MODULE 6 - DESIGN FOR PERFORMANCE, INTERACTIVE MEDIA + PLAY

I. Introduction

So far this quarter, we have primarily explored artworks and design that operate via a one way direction. Films are projected or streamed to an audience, printed materials communicate information to readers, and photographic images are scrolled on a phone or viewed in a gallery or in a book. While engaging with these different types of artworks and media forms can still be a very active experience for the viewer, the form - and how it is communicated - is shaped mostly by the artist.

As we started to explore with video remix and digital media in previous modules, artworks and media that allow for the user to make active choices can influence overall meaning and understanding. This possibility space also immediately begins to blur lines between audience and artist. Interactive artworks and designs - especially ones that are intentionally interactive and playable - allow for more input from the viewer, and can invite an authorship role. The very nature of an interactive piece shifts the viewer into a new role - for this module, these individuals will be referred to as the player-participant. The ways in which player-participants interact with an artwork or design can heavily shape and inform their experience of the artwork.

How much freedom a player-participant has to author their own experience is usually defined by the original artist or designer’s parameters, as they decide which choices and controls to “afford” - or grant / give - to the participant-player. These choices, in turn, can greatly change the ultimate form of the artwork or design.

As with most module topics in the course, interactive and playable media is a HUGE area of study. Multiple undergraduate and graduate programs at UCSC work with interactive media, including Computer Science: Game Design {CSGD}, Human-Computer Interaction {HCI}, Art, Film + Digital Media, and Art + Design: Games + Playable Media {AGPM}. And, as with past modules, there is no way to address all aspects of this field in a week or a quarter or a year or multiple years, so, this will be a brief and focused exploration that looks at a small area of the field, and tries to go a little bit deeper.

So, for this module, we will look at the connections between interactive artworks and earlier works such as collage, along with the impacts of algorithmic technology {pre-digital computational systems} and digital technology on art, design and larger media culture. Part of this exploration will discuss how the process of engaging with interactive systems makes space for player-participants to expand their understanding of complex ideas and concepts, and how this phenomenon has impacted contemporary media culture.

1. Interactive Media Roots

Modern and contemporary interactive media artworks are rooted in several art and design movements, as well as other cultural media, including entertainment media. This chapter will focus on a few of these artistic roots, including Dada-ist collage {it always goes back to collage, right?}, Allan Kaprow’s Happenings, the Fluxus and Gutai movements, and the influence of performance art of the 1960s and 1970s.

These are only a few of many many additional roots, building blocks, and spheres of influence for interactive media, but since this throughline connects back to many of the artworks and movements already discussed in course, this is our point of expansion.

1.1 Dadaism, Surrealism and Chance as Author

Many Dada-ist artworks involved chance or randomness to determine artist choices or certain outcomes. This was further explored by the Surrealist Movement, which directly stemmed from Dadaism. Surrealist artists would often collaborate with one another in order to develop imagined spaces and forms that were often based on randomly continuing drawings made by another artist, without understanding the full form {often referred to as the exquisite corpse}. Salvador Dali, one of the more well-known surrealist painters, experimented with trying to capture visuals and images from the “unconscious mind”, or the moments in between being asleep and awake.

1.2 Allan Kaprow’s Happenings

In the 1950s and early 1960s, artist Allan Kaprow began to plan events and/or performances - which he called Happenings - usually around a very simple premise or set of rules. These Happenings were situations that he designed, and he invited audiences to experience and engage with them, essentially as player-participants.

In some of these Happenings, the audience would explore an environment that Kaprow constructed, such as an enclosed space filled with tires. Other times, the audience was given a specific task to perform based on a set of guidelines or rules Kaprow outlined. In both of these instances, the audience would shift into the role of the player-participant. This type of situational “experience-design” was further adopted and explored by the Fluxus Movement in the 1960s and 1970’s. The images below are of installations of Kaprow’s Yard + Fluids.

The physical spaces Kaprow designed with these Happenings influenced other site-specific interactive sculptures produced by minimalist artists in the 1960s such as James Turrell, Robert Morris + Donald Judd. These sculptures were also eventually installed in galleries and museums, and encouraged audience participation - and performance - with the sculptures themselves. Many of these designs have been reproduced in recent years, and many contemporary artists also work with this kind of site-specific design that encourages audiences to interact with or engage with a physical space in a specific, intentionally designed way. All of these pieces work with both visual design - how the sculptures look and how they are installed - along with interaction design - designing the “rules” of the interaction, constraints and/or limits and what the player-participants are invited / encouraged to do through the possibility space provided.

Robert Morris Installations

1.3 Fluxus + Gutai Movements

The Fluxus Movement originated in the US in the 1960s, and was heavily influenced by Dadaism. The Gutai Movement in Japan originated in the 1950’s, very much in response to the end of World War Two and the introduction of democratic governance - in many ways similar to the origins of Dadaism. Fluxus and Gutai artworks were centered around core themes of exploring situations, challenging expectations for artworks and installation spaces, and deconstructing traditional artistic practices or forms by “breaking” conventional approaches. Many of these artworks incorporate the use of a specific methodology - or a set of rules / constraints determined ahead of time - in order to develop or create a work of art, or define a possibility space for player-participants. These movements overlapped with Kaprow’s Happenings, and developed at the same time with many similar inspirations and influences.

Benjamin Patterson - Performance / Sound / Interactive Artworks - 1960s

Electric Dress - Atsuko Tanaka - 1956

Work - Akira Kanayama - 1957

2. Interactive Media + Performance

Many of the artworks that came out of the movements above could be identified as interactive media. These artworks also inspired or informed other types of interactive media - including playable media, games, and performance artworks - that exist outside of these movements. The sections below cover one of many sets of connections and throughlines.

2.1 Expressive Algorithms

As we have discussed before, Dada-ists worked with more than just 2D images and collage. The Text, Typography + Design Module, showed several examples of Kurt Schwitters’ works, which cut and recombined primarily textual materials in order to created new visual and at times textual narratives. These types of designs are still influential with digital media layouts and visuals. The concept is also very connected to the algorithmic processes driving digital media - where the texts of different narratives can be recombined - sometimes randomly and sometimes according to set of rules, or logic - to create a new set of narratives.

Tristan Tarza’s Dadaist artwork, How to Make a Dadaist Poem, is an collage-like artwork that specifically includes a set of parameters and logic for a player-participant to follow. This artwork itself was not a visual or textual piece - it was instead a set of instructions to form a textual / narrative collage, placing the player-participant directly into a shared author role.

How to Make a Dadaist Poem - Tristan Tarza, 1920

Take a newspaper.

Take some scissors.

Choose from this paper an article of the length you want to make your poem.

Cut out the article.

Next carefully cut out each of the words that makes up this article and put them all in a bag.

Shake gently.

Next take out each cutting one after the other.

Copy conscientiously in the order in which they left the bag.

The poem will resemble you.

And there you are – an infinitely original author of charming sensibility, even though unappreciated by the vulgar herd.

This early interactive artwork was created without computational technology, but still presented a logic, or set of algorithms for the player-participant to work with and within in order to create new media, in the form of a poem. The player-participant could decide some inputs - such as the length of the article, which paper to select and how long to make the cuts / phrases. The overall piece still operated within the constraints set by the original artist-author - selecting an article, cutting words and phrases, placing those texts within a bag and randomly selecting them to determine the order of the new poem.

This type of generative process, which combined a set of parameters with chance operations and player-participant choice and interaction, was further explored by artists across movements and mediums. Minimalist painter Ellsworth Kelly worked with chance operations to drive which colors he selected and where he would apply to paint on a canvas in several of his artworks. Artist Sol Lewitt, another minimalist / conceptual artist, introduced chance through interpretation - he would provide detailed design-documents to produce large-scale painting and drawings, and ask different artists to execute on these instruction-sets. Each painting would be unique, based on the interpretation of the instructions, yet they were all still visually connected through the starting parameters, instructions and constraints designed by Lewitt.

Fluxus artist Alison Knowles designed the artwork House of Dust in 1967, an interactive piece that works with a similar algorithmic approach to Tzara’s How to Make a Dadaist Poem. Knowles generated a series of poems that described an imagined house, and algorithmically determined what each house was constructed with, where it was located, what type of lighting it worked with and who occupied it. Each “house” was output as a unique four-line poem. Knowles would then hold site-specific installations similar to Happenings, where player-participants were invited to gather and interact with the physical construction of one of the poems. This artwork was one of the earliest "computer poems", and the algorithms were executed via an actual computer program.

House of Dust - Alison Knowles, 1967

A house of tin

On an island

Using all available lighting

Inhabited by very tall people

2.2 Possibility Spaces + Interaction Design

Yoko Ono is an artist who was heavily associated with the Fluxus movement at the time, but practiced within her own sphere of individual influence and creative explorations. In the 1960’s, she produced 22 Instruction Paintings: a series of written instructions to be used to create visual or performance based artworks. Some of these works were displayed as only the textual instructions for the viewer to either imagine, or the player-participant to choose to enact. Other works in this series also provided additional components for the viewer to decide to actively interact with, within the exhibition space, such as a hammer and set of nails placed next to a blank canvas.

This type of “experience design”, where the artist crafted a space for potential interaction, was more explicitly explored in Ono’s Cut Piece {1964}. In this performance artwork, Yoko Ono did not provide any specific instructions, and instead set up a possibility-space where she sat clothed on a stage next to a pair of scissors, in front of an audience. This set-up created a possibility for audience-members to decide how to interact with the scissors and the artist - and many often chose to cut away her clothes, even though that was not the only option or something that was directly instructed.

2.3 Contemporary Performance + Possibility Spaces

Viewing the more contemporary performance-based artworks below, consider the ways in which the artist designs the space in which the audience /player-participant navigates. What type of decisions does the artist make, and how do those choices shape or construct the space that player-participants navigate?

Koo Jeong A - OooOoO

Karina Smigla-Bobinski

Solange Knowles - An Ode To

3. Early Games + Interactive / Internet Media

The games and networked technologies presented in this section were not originally considered “artworks”, but were very intentionally designed by authors and creatives to meet specific design goals. I also believe that they were in dialogue with the performance and interactive artworks discussed above, and in some cases, also directly or indirectly influenced by these types of artworks or their impacts on larger media culture. As we have discussed throughout this course, art and design does not exist within a vacuum, and ideas, movements, technologies, and approaches that creatives and designers are grappling with in one form of media are going to at some point be encountered by audiences, viewers and/or creatives engaged with another form of media.

3.1 Early Computer Games - Interactive Fiction

While some of the early visual computer games worked with straightforward action and mechanics, Interactive Fiction games were primarily text-based, and presented players with branching narratives to navigate and interact with via basic written commands and actions. These interactive and branching narratives formally resemble some of the artworks above, such as How to Make a Dadaist Poem, Host of Dust, and the Instruction Paintings.

These types of games were also similar to interactive artworks in the sense that the authors were literally crafting and designing a possibility space - in this case, the game’s world and story - for player-participants to experience via their interactions with the narrative system. Each player would form a unique understanding of that world, based on their chosen actions and responses to the system presented to them.

The text below is from one of the first Interactive Fiction games, Zork, which helped define an entire genre of play. First published by Infocom in 1981, Zork was an interactive fiction system developed at MIT in 1977 by Tim Anderson, Marc Blank, Bruce Daniels, and Dave Lebling. The game revolves around an underground world filled with 100’s of locations and objects used to solve puzzles and progress a narrative through simple commands. When reading through some of the game-texts below, consider the similarities between this computer game and the artworks from the section above, especially in the type of opportunity or possibility space they form through both original authorship and the interaction of player-participants.

As computer technology progressed, games were able to implement visual graphics and sound to construct more detailed, descriptive, and immersive worlds. Contemporary games are also able to provide players with more freedom to navigate these worlds, more complex actions to perform, and more complex systems to interact with - these are still possibilities spaces that the game makers design, usually via a combination of visual, sound and narrative decisions paired with starting parameters, rules - or mechanics / logics - and the choices a player might make. From this breakdown, many games are structurally similar to performance art and interactive artworks, where the original artists designed both how the work might be presented to viewers, and the choices that would be offered to player-participants.

3.2 Interactive + Internet Media

Computational media technologies allowed designers a dynamic way of presenting information or ideas, and designing possibility spaces, but these new levels of interactivity was not something accessible to mainstream audiences at first. While computer and video games quickly grew into incredibly popular and widespread forms of media in the 1980s and 1990s in the US, they were mostly regarded as entertainment, with a specific consumer base and audience.

Within the same 20-year timeframe, computer applications and programs with more specific functions also presented users with a series of choices and options to organize information or data, but these types of media were still relatively limited in reach and use-cases. Some educators also experimented with using computer programs and interactive media to communicate knowledge in different ways - such as interactive CD-ROM encyclopedias or textbooks, or games like the Oregon Trail or Math Blaster - but these were, again, relatively limited in both scope and access. If available, students would play a game or research a topic in the classroom or computer lab, and this experience would be separate from all of the other one-way directional media sources they might encounter.

The internet, websites, mobile apps and network connectivity, and social media platforms are the technologies that brought interactive and playable media to mainstream audiences. These technologies have fundamentally transformed media culture over the past 30 years, and drastic shifts in the media landscape occur more rapidly, with every new development. Digital technology has catapulted the persuasive and expressive power of games, performance, and interactive artworks into mainstream information channels, with massive impacts and outcomes that we are seeing at social, political and economic levels. The next section will explore some theories about how interactivity operates at this level, and why it might have such lasting power on media and culture.

4. New Understandings + Expressive Forms

All of the works viewed thus far in this module explore the role of participation itself, and use interactivity, play, and performance as a way to investigate the role of the viewer and audience, and their impacts on the authoring process when these roles shift to the player-participant.

This shift can be a very powerful method for reaching an audience and communicating complex ideas and meanings. With less explicitly interactive artworks, that are more “one-way” directional, the viewer still can make choices that impact their understanding of a work. There is, however, much less dialogue overall, and the original artist has less influence over these choices.

When working with truly interactive and playable artworks, the artist is able to construct meaningful and intentional possibility spaces for player-participants. As they decide what types of choices, decisions and interaction to make space for, they are able to shape not a single experience or pathway for player-participants, but a whole field or world of possibilities for understanding and engaged and active meaning-making.

This phenomenon has huge expressive and persuasive potential for interactive artworks, and also describes how these types of artworks could theoretically shift the way that we navigate media and understand information. From this perspective, the ideas below consider how interactive artworks and media, made much more prevalent and ubiquitous due to digital technologies and platforms, not only change the ways in which information is presented, but understood. These new capabilities expand what is possible to communicate via interactive art, design and other forms of playable media.

4.1 Procedural Rhetoric

Ian Bogost is a game designer and author who has written extensively about the persuasive and expressive power of games and interactive media. Bogost defined the concept of “Procedural Rhetoric”, arguing that through the process and experience of interacting with a game, players are able to navigate and understand the information and ideas presented via an array of different pathways. According to his theory, this phenomenon can forge new methods for understanding the ideas put forth by the game designer - in short, the process of playing a game can allow players an opportunity to more comprehensively or more fully understand complex ideas.

Potential outcomes are dependent on several factors - including the game mechanics or the rules of play - which are determined by the game designer / artist. Content and narrative are crucial to this theory as well, and work hand-in-hand with the mechanics to drive player understanding. So, if a designer wants a player to achieve a better understanding of complex environmental processes related to climate change through playing their game, this can be achieved both via the gameplay itself, and the game-world. This is very similar to what artists define when developing interactive pieces - the form of the piece, along with the options presented to the player-participant.

The video below clearly explains the concept of Procedural Rhetoric with helpful, straightforward examples. It is required and will be covered by 2 questions on the async lecture review quiz.

I believe this is a very important theory to consider when designing any type of interactive artworks or playable media. Along with re-imagining, there is a huge potential in this type of media to potentially address some of the complex issues we - as artists and designers - are interested in making work about. I also hope - as a creative - that as more and more folks grow accustomed to these types of imaginative frameworks, and are able to understand more complex systems or issues, there will be more possibilities for imagining potential solutions.

4.2 Dynamic Narratives

When describing the concept of Procedural Rhetoric, I like to use an example that shows a possible correlation between the introduction of digital media technology in the US and its impact on media culture and cultural narrative. As mentioned above, interactive and playable media was introduced to mainstream audiences via the internet and websites starting in the late 1990’s, and increasing through the 2000’s with social media, especially accelerating with the introduction of the iPhone in 2007.

At the same time that these dynamic, interactive information channels became more and more accessible, non-linear, dynamic and branching narratives became much more prevalent in mainstream media and entertainment, especially in film and television, and were increasingly popular. These narratives covered many ideas, themes, and topics, and were not only tied to science-fiction or technology-focused content. While there could be many factors contributing to this trend among writers and filmmakers, I believe audiences were and continue to be drawn to these types of narrative because there are still growing numbers of people who utilize interactive, branching media as one of their main sources of information, entertainment and/or social engagement. The way that people interact with information has changed the way we might understand information presented via other forms, showing the resounding impacts that these technologies have had on media and culture.

4.3 Expressive Games

The games below are all highly expressive, and present autobiographical information and experiences to players through interactive experiences paired with each constructed world. These are all examples of using interaction and play to communicate complex ideas and meanings, through intentional creative choices and designs. While each individual has a unique experience of each game, most of those potential experiences fall within the scope of the design defined by the original artist / designer. When playing these games or viewing the trailers, try to consider the layers of meaning that are communicated via interaction and player-choice.

Games by Jenny Jiao Hsia

Jenny Jiao Hsia is a game designer with a library of games based on her personal experiences. Many of these games describe these experiences through direct reference to "everyday" actions and events that communicate more personal information and relationships via the game visuals combined with the rules and progression of each of the games.

Link to Hsia's portfolio below, and play-through the short games Wobble Yoga and Morning Makeup Madness. Then review / play one additional game in her porfolio of your choosing.

CW: some of these games contain cartoon depictions and discussions of dieting and disordered eating - these are noted by the designer's own content warnings on the site below.

https://q_dork.itch.io/

Playing through Wobble Yoga and Morning Makeup Madness is required lecture content. At least one lecture quiz question will relate to these games.

Studio Zevere’s She Dreams Elsewhere

The game in the trailer below is described by the designer as a “deeply personal narrative with a thematic focus on emotions, mental illness, and self-identity”. The visuals and game-play pull from a combination of real-world experiences, inner thought processes, and the creator’s own dreams + nightmares.

Watch the trailer below, and then the interview with the creator Davionne Gooden. This is required for lecture, and there will be two quiz questions based on the interview.

Emma Kidwell’s Half

Emma Kidwell describes Half as “a series of vignettes detailing the experience of being on the fringe of two identities and the invisible toll it takes. Pulled from memories both good and bad.” The resulting narrative is highly personal.

Using the link below, play through Half at least once. Note: this game was having an issue with Firefox on an older Mac, but seems fine with other browsers. Opera is my go-to browser for in-browser games. Note how the artwork of the game relates directly to Kidwell's narrative in terms of color, style and overall subject matter.

https://emmkid.itch.io/half