MFA Thesis Project at ArtCenter College of Design.
Specification
Design Reseach, Speculative Design, VR/AR/MR Design
Execution
Unity, Video, Illustration, Installation, Prints, Interactive Prototype, Thesis Blog, Thesis Paper
Advisors
Tim Durfee, Elise Co, Jenny Rodenhouse, Filip Kostic, Jesse Kriss, Claire Evans
Scenes from the movie Ready Player One
In the current state of immersive tech, you need to push everything outside your peripheral space in order to be fully immersed, and to me that’s really lonely. My interest is in the question: how will this isolating nature shape our relationships with others?
This question led me to a suite of prototypes oscillating between various approaches and mediums. Every investigation demonstrates a different facet of the whole project that seeks to break the self-centric approach.
Alone, Together started with an observation that if you draw with an AR graffiti app, you need to move backward in order to see what's been drawn. By doing that, you could’ve backed up into someone without knowing it. And this is the moment when augmented reality makes your physical reality collide with someone else’s.
These immersive experiences can take place in public everyday when it is popularized by technologies like WebVR, along with the adoption of untethered headsets and ambient computing. To what extent will this situation escalate? And how are we going to design these immersions to avoid people from hitting or bumping into each other?
Implemented with ARkit and laser cut rigs covered with paper prints. Additional insights include: 1) the planes as residues & 2) the layers of planes as information hierarchy.
Inspired by a sentence in my research paper, “AR is constantly looking for planes,” I introduced an “augmented accessory” that the users can wear, which expands a plane around their body. This prototype was meant to challenge how users nowadays are always asked to find a clean surface in order to start an AR experience. With this augmented accessory, the user will have the autonomy of creating a clean plane anytime, anywhere they want.
From this testing session, insights emerged from the participants' behaviors. For example, the users started combining their planes to create a bigger one, or they played with the opposite side of the plane. These insights led me to a series of drawings that reflect on the current methodology (i.e. only the top-side of the AR plane is actionable), and imagine the functions that can be derived from composing the planes in various ways.
One of the drawings indicates two planes overlapping perfectly to create an in-between space. What if that space becomes a place that encrypts some input data? If so, what should human users do to achieve that spatial composition?
These permutations demonstrate potential interactions within a larger group of people. The wiggly forms around the planes are either the indication for possible behaviors of how humans can interact with each other, or the interchange of information/data in these intersectional spaces. If we apply the same concpets of spatial-physical correlations on other permutations of the planes, we can delineate possible applications for decentralized networks, filtering/sharing data, etc.
(Image adapted from Riesenbutzbach. Stage designed by Anna Viebrock)
We can further articulate this relation with VR’s volumentric space, and imagine the correlations between volumes or even between planar and volumentric spaces.
By intermingling these spatial units and speculating how do we perform our body to achieve the compositions, sociality stemming from this technology starts emerging from these moments when we apply this framework to the physical actions in immersive reality experiences.
For instance, when we only hug with people who's very dear to us, that level of intimacy and trust creates a layer of credibility for this relationship that allows data encryption to take place. This proposal provides the digital experience a viseral feedback—the physical interaction. But then questions like what is a genuine gesture, what is performative, or what is the gesture for decryption will arise.
The virtual threshold filters out certain parts of the experience to make the leftovers fit into the social context of the space the user is entering, so every user in that same physical space have virtual components that carries the same material qualities. These users will therefore behave/interact in a certain way, for instance, when a user is transitioning from a VR shooting expereince to an AR doodling space, the machine gun (aggressive) from the fighting scene will disappear or replace/reassign its function with a paint brush or a bubble gun (gentle).
So far, all the scenarios are build upon the premise of users residing in the same experience. But what if they aren’t? Additionally, all the prototypes above are only examining the technological parts but nothing about the human parts, the users. Therefore, Alone, Together starts another thread on user interaction: when we observe the users while they are in an immersive experience, their gestures can be interpreted with some personality, or material conditions. So every experience is assigned to one condition by the way its user operates within, which could be aggressive, calm, rigid, or flexible.
Different conditions react to another one in various ways. For example, an aggressive/spiky experience can rip apart a fragile one. Therefore, by observing the reactions of the virtual world inside of an immersive experience, the user will be able to notice the condition of the outside world and the users occupied in them, whether its an area full of first person shooting gamers, or a crowd that is pondering a virtual sculpture in an augmented reality gallery.
To further explore the idea of materiality, I made a balloon suit to both simulate the volume of a virtual experience and examine how the fragility and elasticity of the suit will change the user’s behavior. The observation that users meticulously slowed down their own actions when they needed to go through a gate/entrance inspired me with the concept of virtual threshold.
In this VR prototype, the user switches through different immersive worlds by physically moving to different locations. Some components from the previous reality follow the user to the next world and stay the same, but some have to change their designated interaction in order to fit into the social/contextual condition of the current experience.
Building on this concept, we can design a system for a city to choreograph different experiences. In the video, different colors represent a zone or a lane, like a highway, that only allows realities that have similar or complimentary conditions to be experienced. When a user wants to switch to another reality, what does a transitional experience look like? Should it be seamless or seamful? Who determines the range of an experience?
Research Project Results & Processes
These questions are informed by the observed behaviors during user testing: many participants traversed back and forth between two experiences. This raised several questions. What are the conditions of the experience that makes them suitable to be placed side by side? Are there exclusions or even monopolies of a certain experience? At this point, this project is moving in the direction of imagining not only the idea of breaking the loneliness, but also examining the design of immersion through the lens of civicness and publicness.
Immersive technology is essentially spatial computing. It takes place in the real world and has physical impacts on people who reside within. When the physical world becomes an operating system, all its users share the same interface. Through the process of observing, prototyping, and iterating, Alone, Together is advocating for the design of the immersive system and its respective interface to escape from the individualistic point of view, embracing the thinking of collective interaction and taking its holistic effect into account.
2018 Media Design Practices Grad Show @Wind Tunnel Gallery
Photo: Sohee Woo, Karina Hernandez