Module 7 - Form, Frame + Framing

The image above documents an artwork by Cory Archangel called Super Mario Clouds. Archangel hacked a NES cartridge, and altered the game to only “play” the sky backgrounds with the clouds. The FRAMING of this artwork, or how it was shown and what was communicated to players / viewers, ultimately changed the way it was experienced and seen in most cases.

7.1 Form, Frame + Framing in Games + Playable Media

This module's lecture content is going to be a bit different than the previous several modules, as it will introduce a few somewhat related visual design topics, but also focus on tying up many of the threads and through lines of the course. The visual design topics explored in this module follow along the logical progression of 2D animation, motion and movement into 3D forms and 3D spaces in games.

This module {and course} will only serve to briefly introduce the use of 3D forms and spaces in games and playable media, as this is a hugely technical aspect of visual game design. While we will not be getting into any of these technical details, there are some important perspectives to consider when working with 3D models and representation that I believe are important and perfect to bring up in this course, as they relate to the culture and social impact of games and playable media.

Additionally, this module will explore a few aspects of how 3D "possibility spaces" in digital games have inspired new forms of gameplay, game interfaces, game-world navigation, and even primary gameplay mechanics. all influenced by a visual aesthetic and perspective.

The majority of the content in this module will be via videos and case studies, that explore the concept of visual aesthetics as a way of "framing" games for players. This will work to wrap up the conversations we've had throughout the quarter about visual design and visual aesthetics in games. This section will explore how different designers form visual aesthetics by combining many of the elements we have discussed in detail throughout the quarter.

7.2 A Somewhat Brief Introduction to 3D Graphics, Environments + Play in Games

Presenting a broad survey of physical and digital 3D art and artworks wouldn't fit well into the pacing and focus of this course, as 3D artworks and 3D games have been covered throughout the module. Many of the visual aspects of sculptural works also communicate meaning via the design principles we have discussed, such as color, light, tone, texture, point, line, shape, scale, textual / symbolic meaning, motion, movement etc. This is also evidenced by the multitude of sculptural artworks presented throughout each module.

Instead, this module will immediately delve into the use of 3D graphics and 3D worlds in games, addressing some historical touchstones, and then move into some important more social and cultural considerations when working with 3D models and characters. This is primarily because working with things like color, texture, lighting, construction and movement in 3D forms gets into technical areas that are outside of the scope of this course, and especially in what we can explore with this module's project {which will be discussed more in class}.

Early 3D Roots

Video and computers games have worked with the concept of "open worlds" or 3D environments for several decades and really since their origins. Early text-based games and interactive fiction such as Adventure presented players with a sense of large, descriptive game-worlds and many options for interaction and play - they just happened to be textually-based.

The game Elite, published in 1984 for the Acorn Electron computers, was possibly the first game to use graphics to communicate the sense of navigating a 3D world. These graphics were minimal and wireframe, but did indeed offer a sense of navigating space in order to explore, resource mine and trade in an expansive universe with procedurally generated planets - this also could be considered a starting point for modern "open-world" games.

A quick general content warning for this section 7.2 - a few of the images under the first person and third person games contain a few low resolution, “cartoon-like” depictions of blood and guns. This is nothing extreme and at most would be rated PG-13, but just wanted to give folks a heads up.

In the early 1990s, several games started exploring the idea of 3D play in a few different ways. The Terminator {1991} allowed players to navigate a basic polygonal 3D city - either from a first-person perspective or within a vehicle - intercut with more cinematic puzzles and navigation - such as reading through a phonebook, selecting items from a storefront or jumping to location via a 2D map. The combat elements in this game were relatively slow and clunky, especially when compared to the game that many considered the first "3D shooter", Castle Wolfenstein 3D, which came out for PCs only a year later in 1992. This game utilized 2D sprite characters in a proto-3D space with a first-person perspective, which allowed for much more responsive action, animations and detailed visuals.

Specific Content Warning - Castle Wolfenstein 3D takes place in a fictional castle in 1930’s or 1940’s Germany, with the player character shooting Nazi soldiers and trying to escape. There are a lot of Nazi symbols and imagery throughout the game, but not in the images below.

The First First-Person Shooters

Castle Wolfenstein 3D’s visual aesthetic and gameplay was further developed and explored in Doom {1993}. Doom enhanced the 3D-like environment by adding levels and a sense of vertical depth and scale to its environment via platforms, stairs, towers and sunken floors, added lighting, shading, liquid, and other atmospheric effects, and additional player character controls such as jump and crouch. This game, and its direct descendant, Quake {1996}, which incorporated real-time 3D rendering and effects and networked multiplayer options, along with Unreal {1999}, are considered by many to have defined the First-Person Shooter genre {FPS}. Their influences are still apparent in modern FPS games like Call of Duty, Apex Legends, Overwatch, Half-Life, Halo, etc.

Third-Person Perspective Origins + Innovations

At the same time that the tgamesabove were exploring first-person perspectives and gameplay in proto-3D or true 3D environments, other PC and 3rd gen console games were working with a 3D aesthetic that lay the groundwork for open-world games like Zelda: Tears of the Kingdom or Red Dead Redemption 2.

Alone in the Dark {1992} and Ecstatica {1994} were both third-person perspective survival/horror games that worked with 3D polygonal characters and objects navigating pre-rendered environments. Multiple cameras would be set up in fixed locations throughout the pre-rendered scenes to frame the player's perspective, and would automatically switch based on the player-character's position.

This format inspired many of the early Resident Evil games, and informed the more contemporary third-person approach of having the player's perspective framed via a camera that would follow and center the player-character, with the ability for players to somewhat control it. This approach was effectively used in Tomb Raider {1996}, which allowed players much more freedom to navigate and interact with a more dynamic 3D rendered environment, even though that environment was still mostly interior spaces. This strategy allowed the designers to craft a sense of an "open" 3D environment that players could fully navigate, but was still contained by "natural" features, such as walls, waterfalls, canyons, steep cliffs, etc.

The console games Mario 64 {1997} and Zelda: Ocarina of Time {1998} continued this expansion of what was possible with third person action and gameplay in 3D environments and worlds. Zelda: Ocarina of Time also explored "open-world" concepts such as night and day, and the passage of time. Both games, but especially Mario 64, also worked with a more highly stylized visual aesthetic instead of aiming for a sense of realism and or high-fidelity, like Tomb-Raider or Quake. This was something that was very unique for the time, as most other 3D games aimed for realism and detail, while Mario 64's characters and worlds were, comparatively, lower-fidelity, but still incredibly expressive, fantastical and immersive.

Grand Theft Auto 3 {2001} implemented a larger, navigable 3D cityscape that players could always access regardless of narrative progression. This game's third-person perspective, combined with a 3D open-world, more fully defined the contemporary "open-world" or "sandbox" gameplay concept most often associated with 3D aesthetics and explorative and/or action-based 3D games.

First-Person Adventures

Finally, no historical survey of 3D forms in games and playable media would be complete without discussing Myst {1993}. This game defined the "first-person", point and click adventure puzzle game, not via a true 3D world or engine, but by creating an immersive gameplay environment via “3D” still scenes, tied together by pre-rendered "walk-animations" that bridged one static location to another. At the time, these pre-rendered visuals offered a much higher level of detail and world-building than any games with actual 3D aspects, making the game incredibly visually compelling and popular. Myst also inspired an entire genre, informing games like What Remains of Edith Finch, Gone Home, and The Witness.

When reviewing the visuals below - and the reading for this week's module - be sure to consider that it is now completely possible to turn the original Myst into a real-time 3D game, with even higher levels of visual detail and effects. How might this change the game for both players who have never experienced the original, and those that did?

7.3 Embedded Bias in 3D Character Designs, Models + Rendering Tech

As we have seen above, 3D games' visuals were originally severely limited by and mediated by technological constraints, both in terms of production, visual output and the computing / processing power needed to rendered 3D objects and environments in real-time. These technological constraints, currently, have been vastly minimized - huge amounts of details and effects can be rendered, adjusted and manipulated in real time.

Just because there are an infinite number of visual possibilities that can now be rendered with precision and authentic, artistic perspective, does not mean that this is happening for all subjects in 3D games. Below is a short video featuring artist, designer, and UCSC Professor, A.M. Darke, introducing her current creative endeavor, the Open Source Afro Hair Library {OSAHL}. This library was started as an "intervention in digital media", disrupting both the lack of representation and the misrepresentation of Black character designs and models - especially in terms of hair texture rendering and hairstyle models - in many mainstream games. The project also addresses the commodification of Black models available for purchase via 3D Asset Marketplaces, and instead offers artists and designers, working within the OSAHL creative license, open access to the designs, as well as the coding, implementation, and style guides for working with the models.

The video below is required to watch, but is not covered by the lecture review quiz.

Much of the reading for this week delves into Professor Darke's work on this project, so, I will not go into too much further detail here, but I do want to emphasize that this is a critical issue that connects back to the use of fixed-palettes in Module 3. That module discussed how the Nintendo Entertainment System’s fixed and limited color palette made it difficult for game designers to incorporate characters with darker skin tones and complexions, which vastly decreased the number of characters - especially player characters - with darker complexions. In both situations, the technology itself is embedded with the bias’s of the designers, and it is limited in ways that exclude representation or reduce creative access and expression for many players and designers.

Along similar lines, using current 3D modeling and rendering technologies, it is totally possible to form, represent, and render Black hair textures and hairstyles realistically and diversely in ways that communicate a sense of Black visual identity. This realistic and authentic representation, however, is something under-prioritized by mainstream 3D production programs in pre-defined settings, demonstrative tutorials, features, and other included asset libraries. In contrast, many of these same programs will highlight to users how to implement realistic movement, visual texture, and other effects for non-Black models' hair. This lack of creative access and experimentation then limits expectations of what Black representation in games could and should look like, further limiting character design possibilities and further perpetuating harmful visual misrepresentation and stereotyping among non-Black art directors and decision-makers at an industry-level, which then funnels down to non-Black players.

OSAHL brings together Black artists to experiment with models, texture-effects and designs, and intentionally shares visual discoveries and models with other game designers and 3D artists. The readings will discuss this project more in depth, but for now, be sure to watch the video below and check out the OSAHL website, go through the About and License sections, and explore a few of the different models featured.

https://afrohairlibrary.org/

7.4 Form, Space + Frames in Games

3D graphics technology and systems allowed games to explore an entirely new forms of visual aesthetics, and player experiences. While the affordances of 3D technology often focus on forms - such as detailed environments, characters, and effects - this technology also opens up possibilities for the inverse of forms - space.

Space can be hard to envision and describe in games. One method I have found useful for identifying games that work with space in unique ways is to look for games that center a core mechanic around unconventional navigation and/or shifting player perspectives / framing.

Portal {2007} was one of the first 3D games to fully center gameplay around shifting spaces and framing in this way. Set in the Half-life universe, the game focused around using portals - or engineered "breaks" in the game world - to solve puzzles and navigate through a space. View from a formal perspective, this game presented players with a way to think about, visualize, manipulate, and explore space from an entirely new perspective, unique to both games and other interactive artworks and visual media.

Other games that work with space in unique ways include The Stanley Parable, Superliminal, Manifold Garden, and the very recent Viewfinder. Navigating the play-space in unexpected ways allows designers the opportunity to incorporate dynamic narratives into games, and also makes space for working with collage-like visual aesthetics within the game-world itself. This connection is especially apparent in Viewfinder, where the action of collage is what triggers the shifts in space and frame, and completely shits the narrative experience for players. I believe it is very interesting to consider how in these cases, the use of a 3D play-space can influence mechanics, which in turn further influence the visual and narrative aesthetics of a game.

The video below is not required, and contains spoilers for the game Viewfinder, but is also very interesting and shows many examples of the game’s unique use of space and framing.

7.5 Framing, Game-Cameras, and Cameras-as-Games

Most first person and third person 3D games work with in-game cameras to frame the viewer's perspectives as they navigate the world. In first-person games, this camera often represents the player-character's perspective, while in third person views, this camera frames the player-character in view. Both of these viewpoints often allow the player to take control of the camera and shift the framing or viewpoint. This is an interesting aspect of visual aesthetics and design in games, as the ability to shift the camera can impact the framing of the player experience, and introduces more of a possibility for player-authorship and incorporating a player's own artistic expression or creative viewpoint into games.

There are also some games that center around the camera, and/or documenting player perspectives and framing, as a core mechanic or aspect of gameplay. Viewfinder works with the idea of player perspectives via a camera and photographic images very directly. Games like Pokemon Snap, Life is Strange, Firewatch and Season: Letter to the Future, utilize framing via an in-game camera in similar ways.

Umurangi Generation

Umurangi Generation is a photography-driven game that is compelling for it's unique photo-focused mechanics, and its visual and narrative aesthetic. Created by designer Naphtali Faulkner, the game's narrative, characters, environment and visual style are influenced by Faulkner's Ngai Te Rangi culture. Many of the narrative components were also informed directly by the Australian government's response to the 2019-2020 bushfires, as well as the Covid-19 pandemic, and more broadly address themes of environmental, political and social collapse driven by capitalism and colonialism. I believe games like these represent a new direction for meaningful play and creative player experiences, as they create a space for a player's own expression and art-making within the designer's constructed visual world. Players can select different camera settings and lenses, allowing for another level of authorship within the game play.

7.6 Framing Perspectives

3D games - either first or third person - often need to present and communicate a large amount of information to the player in ways that can be easily read, understood and responded to. Additionally, 3D environments and gameplay will usually require more complex player controls, at least in the form of navigation if not a range of additional interactions with a dynamic environment. For these reasons, User Interfaces - especially in the form of "Heads Up Displays" or HUDs - in 3D games play an even more critical role in the overall gameplay aesthetic. The effectiveness, cohesiveness and overall functionality of these designs is also heavily tied to their visual aesthetic, and how it works within the larger game's visual aesthetic.

These interfaces also, essentially, "frame" and inform the player's perspective. They influence and/or shift their understanding of the the gameplay, communicate additional layers of meaning, and, at times, provide additional pathways for navigating narratives and other gameplay mechanics. Interfaces - especially more complex ones - therefore can also be defined as "framing devices" in games. In some games, these are the actual points of interaction / connection between the player and the game-world.

7.7 Visual Aesthetics as a Framing Device

Tyeing up the threads of the last 9 weeks, this last section proposes the idea of a game's visual aesthetics as a "framing device" for players to best understand and/or experience the gameplay and overall play aesthetic. Working from this perspective, the visual aspects of a game cannot be separated from its' gameplay, and, if designed intentionally, can also provide a straightforward path that designers can shape to ensure the most access to, and connection with, their intended gameplay experience. Not every player is going to have the same play experience with the same game, but a defined, expressive, cohesive and communicative visual aesthetic can have a huge impact on informing, shaping, reinforcing or possibly redirecting those experiences in ways that mechanics and dynamics alone cannot.

The videos below look at three different approaches to defining and implementing visual aesthetics into games, and how these visual aesthetics can frame overall gameplay for players. These are all unique, and varied, and present different ways for thinking about visual aesthetics in games.

Jenny Jiao Hsia The Aesthetics of Cute

The video below is required to watch up until the end of the talk and the start of the Q + A {about 30 minutes}. Content / Trigger Warning: from minute 25 to minute 29, the artist talks specifically about her experiences with dieting, and how she envisioned originally incorporating that experience into a game. It is totally ok to skip this part if needed.

This video is covered by two questions in the lecture review quiz. These questions can be answered without watching the video between minute 25 and minute 29.

Ben Esposito - The Design of Neon White

This video is required to watch. It is covered by two questions on the lecture review quiz.