Playing with the Here/Now/There/Then:

Crafting Authentic Hybrid Experiences for a Networked Culture

1.0 Introduction

We live in a networked world with increasingly blurry lines between our physical and

virtual lives. Our computation-centric society is more orderly, more systematic, and

more governed by algorithms than ever before. Technologists often rush forward, in the

name of progress and productivity, without looking back to reflect upon the effects (or

missed opportunities) in their wake. Focusing on the “how” instead of the “why” these

new pioneers develop faster and better ways of systemizing, quantifying, streamlining,

documenting and tagging our world objectively. As new algorithms get pumped out

faster and faster, ubiquitous networked computing pervades everything from our built

environment to our pockets.

In contrast to this rushed speed of progress, I believe in a more mindful yet exploratory

approach to developing technology, in order to create idiosyncratic relationships

between humans, computers, and the network. Instead of seeing a spacial or

technological constraint as a potential problem to be “cured” away with technology,

I see it as an opportunity to use play as a tool for challenging existing systems. I am

specifically interested in systems which enable uniquely subjective hybrid experiences

which are made possible by leveraging the real world, the virtual world, and the human

imagination. For instance, the physical world provides the benefit of tangibility while

Jayne Vidheecharoen

1

the virtual world provides the collaborative power of our network culture. And the

human imagination provides the cognitive mortar which holds everything together and

allows a user to create meaning from the composited experience. Due to their hybrid

nature, these experiences have their own unique sets of constraints and therefore their

own unique opportunities for disruptive play. Using play, I aim to create a new hybrid

experience while provoking questions about existing interaction norms, both online and

in real life. In the end, I am hoping to create a new space of possibility for others to

build upon and expand.

2.0 Work & Play: Magic Circles

Our future technology will not only enable us to work more productively, but in fact

more playfully as well. Ubiquitous computing has created a fundamental shift in the

way we separate work and play (Hilbrecht). The idea of work was once confined to the

tangible real world, while play was allowed to exist in the intangible imaginary world.

Now for many of us, much of our daily labor exists only in a virtual, intangible, and

imaginary realm. As a result, this liberates our daily tasks from the rules of reality while

subjecting them to the rules of play.

My early experiments, Zombie Mail & Play Slideshow, challenge the standard

environments of banal digital productivity software to be more playful, while

questioning established user interface norms. We often spend much of our work day in

these dull virtual spaces, but being completely virtual they are especially viable

Jayne Vidheecharoen

2

candidates for becoming magic circles. Johan Huizinga, the classic expert of play in

culture, defines a magic circle as a special temporary play space within the ordinary

world where rules enable specific performances. In this case, having been traditionally

designed for the rules of productivity and efficiency, these environments enable a

particularly objective and task oriented performance by the user. The modifications in

Zombie-Mail create custom metaphors for the e-mail interface, on both a visual and

interaction level, as a way to reflect a users’ subjective feelings about their experience

with e-mail. It also favors a more discovery based navigation as opposed to a

traditional list based navigation. Unlike a mere skin applied over an existing interface,

this navigational shift fundamentally changed how the user interacts with the

information and possibly their attitude towards the task at hand. Similarly, Play

Slideshow makes the normally mundane task of creating slideshow presentations more

challenging and playful by adding constraints. In this case, elements can only enter the

slide by dropping in from above while the user is given limited controls (as in Tetris). By

tweaking the interaction it begins to draw attention to the repetitive and homogenous

act of creating slideshow presentations.

Not only does play, having its own rules of space and time, make something fully

absorbing for a player it can also be used to expose tensions within a system. Since

challenging a tension or established system is consciously outside of ordinary life (a

formal characteristic of play), outlaws and revolutionaries have an inherent element of

play in their activities (Huizinga). Thus, being playful is a potentially revolutionary and

disruptive way to change established systems.

Jayne Vidheecharoen

3

3.1 Constraints & Systems: Activating Public Spaces

Creatively and playfully surmounting the constraints of a rational system is an

opportunity for intervention and improvement. At the urban scale, people who reclaim

public spaces to suit their own specific desires are defying the design of these urban

systems, while actively challenging the way these spaces have been scripted by

architects and urban planners. Margaret Crawford notes that mutability, contestation,

and change is not what constitutes a failure of public space, but instead defines it. The

emergent activities in Los Angeles’ public spaces are the very things that enable us to

pose questions about urban citizenship. Therefore, we should not mourn the loss of

our public spaces but instead see it as a space filled with possibilities (Crawford).

Although it exists in a completely virtual space, Google Street View is essentially a

massive simulation of real world public spaces, frozen in time and filled with un-tapped

potential. Through many of my experiments I’m specifically looking at ways to bring it

to life and activate the space so it can be more like a real public space, subject to

mutability and idiosyncratic uses. First Person Creator questions the use of Google

Street View as a purely static and historically accurate tool for viewing an archive of our

real world. While Google Earth allows people to contribute 3D models of buildings and

monuments, Google’s strict guidelines only accept models which represent real world

structures textured with photographs. Therefore, it is impossible to contribute

innovative or imaginative structures to the collective archive. In contrast, First Person

Jayne Vidheecharoen

4

Creator imagines a sandbox-game style interface which allows the user to augment

and interact with Street View as they desire. The first person perspective is also a shift

from the section, plan, and birds-eye-view of traditional 3D modeling programs. This

perspective empowers people to feel situated within the space, enabling a sort of

spacial play testing. Other mash-up-mock-ups, such as Dancing in the Street View and

Rolling in The Street View, continue this line of enquiry and begin to expose potential

creative opportunities for adding life, through animation, to this static virtual public

space.

3.2 Constraints & Systems: Improvisational Alternatives

In the physical world, Parkour is a clear example of both a form of play and form of

resistance to the scripted use of urban systems and spaces. Nathaniel Bavinton

explains that for Parkour enthusiasts, known as traceurs, constraints do not hinder fun

but in fact add value and enhance it. Thus creative improvisation comes not from

removing the constraints but by reinterpreting and finding new possibility within the

constraints. This is particularly important to remember when considering the role of

technology, which has historically been focused mainly on removing obstacles to make

our lives easier. I am experimenting with what this approach of improvisational

interactive play might look like when we substitute the traceur’s buildings and railings

for the technologies we are given as consumers.

Jayne Vidheecharoen

5

Users are not merely pedestrians adhering to a set clickable-path but instead digital

traceurs who can be empowered to gracefully transcend technological obstacles

through a wide range of embodied techniques. One of the key features of Portals is the

ability to bypass the hurdle of the screen by being able to (seemingly) interact with the

interface inside the screen. While touch screens and gesture based interactions already

allow for a more embodied interaction than the traditional keyboard or mouse, in both

cases the screen still acts like a glass wall between the user and the interface. Portals,

on the other hand, is able to serve as a potential alternative to what Bret Victor calls

“Pictures Under Glass” technology. “Pictures Under Glass” is a predominant vision for

interfaces (in both the present and future) which forego the richness of tactile feedback

in favor of high resolution visual representations. By looking for a fun and fluid way to

both bypass and take advantage of this constraint, I propose an alternative approach

to digital interaction, which is capable of leveraging both the tactile and visual senses

in order to create a more authentic simulation of interaction. The interaction in Portals

may not be easier, as the system tends to cause the user some spacial confusion, but I

believe this additional obstacle adds to the experience by making it more fun.

4.1 Truthiness: Believe

The idea of truth is another rational system open to exploitation & intervention as a way

to create future possibilities. Questionably objective truths, created by algorithms,

already shape our real life in many ways. In his TED talk, Kevin Slavin argues that the

repetitive nature of algorithms enables them to gain the sensibility of truth (even if

Jayne Vidheecharoen

6

they’re not true) until eventually they solidify and become seemingly “real”. For

instance, in the 2010 “Flash Crash of 2:45” we saw algorithms taking control of the

stock market, dropping the Dow Jones Industrial Average about 1000 points before

recouping the losses minutes later. Despite the fact that no human had ordered it or

had any control over what was happening, the effect of the crash was very real.

Code even controls the small decisions we make in our everyday lives. Netflix’s

recommendation algorithms are responsible for 60% of the movies we end up

watching (Slavin). Unlike the code crunching quantifiable financial data, the Netflix

code is essentially turning our subjective relationship with art & culture into something

seemingly objective and true, which influences the way we spend our time in real life. If

algorithms have the power to make us believe in a truth that affects our lives, as

individuals we too have the freedom to play with the truth in order to affect our own

lives.

4.2 Truthiness: Fictional Facts

A subjective and “loose” definition of truth, when used in cahoots with the viewer, can

become a valuable tool for creating a more “real” experience of the present. This can

also allow for the viewer to create their own interpretations, beyond the creator’s

original intentions. P.T. Barnum was infamous for playing with the expectation of truth

in the context of museums. He often manipulated the press to stir up controversy

about the authenticity of his own Dime Museum exhibits. This drove up ticket sales

Jayne Vidheecharoen

7

because he knew the audience loved trying to figure out if the exhibits were fact or

fiction. By inviting the viewers to question his exhibits, he was respecting the viewer’s

intelligence. Many of the viewers enjoyed collaborating on the “intellectual exercise”

regardless if it could be proven to be true (Dennett). The development of 19th century

technologies also left people thinking that just about anything mechanically or

organically probable was possible, making people very susceptible to hoaxing (Harris).

This seems especially relevant in our modern times, given our dependence on the

internet as a main source of knowledge in addition to the rapid pace of developing

technologies which were once only fantasy, such as ubiquitous networked computing,

and complex predictive algorithms.

The narrative I create in my Kickstarter campaign describes Portals as gateways to

another place in the world, at a different point in time, even though they’re clearly not

actual portals with teleportation capabilities. The description also states that it allows a

user to go “inside” the screen, despite the fact that the user is technically behind the

screen. I specifically craft a description which is both true and potentially fantastical. I

also include videos which are just rough sketches of how the Portals could potentially

work. Some see these videos as examples of completely functional prototypes while

some question their verity. Given the honor ecosystem of Kickstarter, it is especially

interesting that this clearly fictional idea of going into a screen to access portals to a

parallel world is an acceptably honest and authentic description of a technology

project to backers. Like P.T. Barnum, I feel it’s important to respect the backers’

intelligence by inviting them to knowingly play along with me in crafting the projects

Jayne Vidheecharoen

8

story and advancing it further. As a result, various blogs continue the story with factual-

sounding headlines like “Media Design Student Reaches Into Another Dimension” (Art

Center Dotted Line), “Portal Becomes A Reality--Sort of--With This Kickstarter

Project” (PC World) and “Thinking inside the box: The television you can 'reach inside'

to grab things on screen” (Daily Mail). Clearly, playing with people’s imagination is a

potentially powerful point of reference for people to grasp a new idea.

4.3 Truthiness: You’re Doing It Wrong

It is also through challenging the true “official” intended purpose of a system or object

that we can create a new possibility space for things to enter the world. McKinzie Wark

explains how hacking is essentially abstracting, and it is through this act of abstraction

that a new space for possibility, beyond necessity, is created. By liberating objects

from their designed purpose, it becomes possible to explore new uses. Within my

Portals project I consciously explore the “wrong” ways to use technology (on both the

software and hardware level) as a way to provoke questions and speculate about what

another “right” way might soon be. While my “wrong” uses of technology may seem

strange at the moment, in the future it may become second nature. Some blog

commenters don’t see any practical use for Portals while others are excited about the

range of potential applications. Several Kickstarter members have contacted me

excitedly with additional application ideas such as table top gaming, race training,

presentation tools, and educational tools. Despite the fact that I’m using everything

“wrong”, to these people, my system feels like the “right” solution for their goals. It’s

Jayne Vidheecharoen

9

important to give people enough context to understand the main ideas behind the

project, while still leaving the space of possibility open enough to allow “wrong”

personal trajectories of inspiration from the initial idea. Hopefully this ensures that the

project is the start of a conversation as opposed to a definitive solution to a single

problem.

5.1 Hybrid experiences: Alternative Reality

As more of our experiences exist in a hybrid reality we are increasingly open to

enhancing our “real” experiences with an “imaginary” virtual layer. We are already

comfortable calling upon “the cloud” to augment our experience of the real world.

Whether it’s changing our behavior to earn a mayorship on Foursquare or letting Yelp

determine our night out, we are perfectly comfortable with letting this virtual layer

permeate our real lives. We can therefore freely borrow from techniques of traditionally

virtual spaces as a way to enhance our real world. Harnessing the power of mixed

reality enables us to challenge our expectations of what is possible in the real world.

Jane McGonigal argues that we can “fix” reality by employing tactics used in games,

such as inviting the user to take on unnecessary challenges, in a personally meaningful

way. She defines alternate reality games as “games you play to get more out of your

real life, as opposed to games you play to escape it.” These alternate reality games are

especially powerful because they give us explicit permission to play outside of social

norms within our real life.

Jayne Vidheecharoen

10

In order to test McGonigal’s theory I created Media Design Bingo, an alternate reality

game designed to increase interaction between all the students in the studio. Other

design goals included designing a game that promoted specific values, such as

gratitude and collaboration, to hopefully encourage a wider range of people to play.

Using the studio floor plan as the bingo board, people were asked to keep track of the

people who had helped them throughout the week, in attempt to complete patterns for

badges. I was surprised to find that people did indeed feel the game increased the

level of interaction within the studio for the week, and people felt more comfortable

approaching some peers they hadn’t in the past. Clearly, this alternate reality game had

created a sort of magic circle where the rules of the game enabled new types of

interactions to occur and actually showed how play could be used to address and

change real life tensions.

5.2 Hybrid Experiences: Movie Magic

While alternate reality games can be powerful, the gamification of real life is only one

type of hybrid experience. If we think beyond gaming and include other forms of “fake”

worlds, such as film, we find another rich set of tools for creating engaging immersive

experiences. The imagined virtual space also has a rich tradition of leveraging

technologically mediated illusions to create the suspension of disbelief, or the

suspension of reality-testing all together. The world of animation, compositing, and

special effects have advanced so far in the past decade that often times we can no

longer separate between what was shot in-camera and what was added in post-

Jayne Vidheecharoen

11

production. As a result, the distinction between the real and the fake dissolves as we

allow ourself to become fully immersed in the experience of the film.

Illusions are powerful because they trigger a feeling of authenticity which can be

manipulated and reshaped (Stark). With Portals I am consciously embracing “movie

magic” techniques, such as chroma keying and layering footage, and applying them to

the context of a real world interface. As a result, the user experiences a simulation of

interacting with the interface directly, while taking advantage of the screen’s ability to

augment and animate. Stop motion animation in particular, is an especially powerful

tool because it has a realness in terms of materiality and motion, but it can still be

stylized beyond what is possible in reality. As a medium, stop motion is inherently

about illusions caused by playing with time. Due to it’s very nature, it also highlights the

uncanny tension between the unexpectedly static or animated when in the context of

the real world. By combining illusion and interaction I strive to create a more magical

experience for the user, which takes advantage of both the real and virtual world, to

create a new type of hybrid space.

5.3 Hybrid Experiences: The Real World

Of course, the hybrid experience goes both ways, in addition to drawing from the

virtual world we should enhance our virtual spaces by drawing from the real world.

With Portals, the ability to physically manipulate objects in the interface takes

advantage of our hand’s sensorial feedback capabilities. Portals’ reality based visual

Jayne Vidheecharoen

12

interface also helps grounds the hybrid experience in the real world as opposed to only

having a place in a virtual one. While virtual spaces like Second Life allow users to

interact with each other as CG avatars in the same virtual space, they also only exist in

a completely fantastical environment. Despite early hype around the potential use of

Second Life as a legitimate alternative to in-person meetings, the idea failed to gain

mainstream acceptance. Relying completely on CG, everything about the world

completely disregards reality and becomes stuck as an escapist amusement, as

opposed to one which could potentially enable the user to benefit from the activity in

their real life.

On the other hand, standard video chat allows for a completely reality-based

conversational experience, but it’s limited in that each person is confined to their own

little window of reality. It also completely ignores the potential for a more fantastical

experience. While visually “real,” the experience of video conferencing distinctly lacks

the ability to really co-exist and visually overlap in a shared third space and play

together. Alternative solutions such as sharing a desktop screen or using a

collaborative white boarding application still leaves the human element stuck on each

side of the side of a screen. The only hint of a human’s presence in these virtual

workspaces is reduced to a wiggly mouse cursor, and maybe a chat box on the side.

Tangibility is just one of many benefits of the physical world, another is the ability to

just co-exist in the same physical space with people while interacting with them.

Portals strives to exist somewhere between virtual worlds, shared desktops, video

Jayne Vidheecharoen

13

conferencing, and the real world in order to challenge our standard manifestations of

co-presence.

6.1 Network Culture: Strength in Numbers

While striving to create hybrid experiences that explore new modes of telepresence,

network culture becomes a powerful tool for generative, collaborative, and distributed

creation. Kazys Varnelis sees our current network culture as a distinctly new

phenomenon, defined by the mix of real and virtual space and new forms of

participatory media. While the effects this new culture seem minor to each of us as

individuals, collectively it’s a radical shift. In the network, individuals become less

important than the emergent outcome of the connections between people (and

machines). Naturally, the process of remixing from many sources is the dominant form

of creation in this highly interconnected culture, dissolving ideas of individual

authorship (Varnelis). As such, experimenting with the affordances of network culture

within my project is very timely and highly relevant to our current experience,

particularly in relation to remixing real and virtual spaces and participatory media.

In many ways, Media Design Bingo is a tangible manifestation of our networked culture

in the real world. The individual game boards log the help each student received from

other students, essentially serving as a visualization of generosity within our real world

social network. Additionally, customizing the individual boards is explicitly encouraged

in the rules of the game, giving people freedom to freely express themselves. As a

Jayne Vidheecharoen

14

result, the ways in which people chose to customize their individual boards combined

with the network activity generates a collective image of the studio culture that can’t be

predicted by any individual or an algorithm. The unique award badges created at the

end of the game also serve as a visualization of the connections between students,

demonstrating the emergent outcome of the network. Testing the capabilities of crowd

sourced creativity more directly, I create a HIT (Human Intelligence Task) for Amazon’s

Mechanical Turks and pay 20 people $0.25 each to modify a panorama. The space

evolves as various people from all over the world playfully add their modifications and

alter the modifications of others. While the test is relatively small and individual

modifications are minor, over time it becomes quite lively and demonstrates how a

system of distributed creation could work. I imagine that if Portals were widely available

(and fully featured) they would enable a similar kind of emergent and collaborative

activity. Launching and running a Kickstarter campaign to raise funding for Portals also

serves as another form of public research into our current networked culture and the

power of the internet. The 30 day campaign has ended with the project 190% funded

by 118 backers, with $2,853 pledged. About 35% of the funding comes from my

personal network and their extended networks, with the other 65% of funding from

random strangers on Kickstarter. These results clearly demonstrate our desire to

participate and help each other through the network as a collective.

Jayne Vidheecharoen

15

6.2 Network Culture: Uncanny

While our network culture creates a whole new range of possibilities, it also creates an

experience of tension within our minds as we reconcile the experience of transcending

space and time. When we can get updates from the other side of the country at the

speed of a tweet and travel the world with a drop of the Street View Pegman, our

concept of the here and now blurs with the there and then. Through my experiments I

intentionally aim to further muddle the distinction in order to create an uncanny

experience for the user. Kio Stark explains that playing with the uncanny is a

particularly powerful tool because of its ability to cause people to question a

fundamental understanding, even if only momentarily. These uncanny experiences

often cause us to suddenly reinterpret our experience using “primitive beliefs” such as

magic or coincidence. But we are less likely to feel like something is uncanny when we

are more familiar with our environment. As we become more and more accustomed to

living in a network culture we run the risk of becoming immune to the strangeness of

our experience. Thus, provoking an uncanny feeling can serve as a way to jolt us from

our daily routines as a node in the network and cause us to question the things we

currently just accept as normal.

In order to create an uncanny experience, I explore alternate ways to show different

points in time within a single space in my panorama mashups. In Pan, looking around

horizontally reveals a different point in time each time the viewer returns to the space.

Scrub, on the other hand, stacks time vertically like a film strip so the viewer can see

Jayne Vidheecharoen

16

one slice of a space over time while also being able to look around a space at a

specific time. Both experiments explore the implications of being able to interact with

and remix a time-based media (for instance animation or film) within the context of a

traditionally static environment. As these extra layers of reality co-exist with a static

representation of reality, the user is able to weave their own non-linear understanding

of the space. Similarly, in Portals the layering and compositing of static images from

the past, footage of the present, and animated interactive stop motion creates a

collage of multiple realities. The collage is by multiple authors and is neither real-time

nor pre-recorded. By creating an uncanny experience of space and time I also strive to

draw attention to the way in which our “authentic” representations of real space, in

virtual environments, are actually stitched together from various moments in time,

breaking the ties between space and time. In doing so I am suggesting alternative

illusions of virtual co-presence that are equally open to a non-linear time structure.

7.0 Conclusion: Starting Principles

From the theory and experiments conducted thus far I have developed several

principles to guide my work moving forward. First, by leveraging spacial and technical

constraints, use play as a tool to disrupt and challenge existing magic circles in the real

world. Through this act, instigate more improvisational interactions between both

human to human and human to computer. Second, embrace alternative “truths” as a

tool for sharing ideas and inviting participation from others. By provoking questions

about future technological developments, hopefully this technique ensures the project’s

Jayne Vidheecharoen

17

core ideas can continue exist beyond its original “actual” form. Third, combine the

affordances of the virtual world, the physical world, and the human imagination in order

to create an experience formerly impossible in the virtual or physical world separately,

and therefore more authentic to the hybrid nature of the experience. Finally, using both

the strength and strangeness of network culture, create an uncanny experience in

order to disorient users and disrupt our habitual routines as active participants in the

network.

With these core principles in mind, I feel well prepared to make progress on my thesis

project through the Spring term. For the duration of the term, I intend to focus

specifically on developing Portals as a system. This includes the design of the objects

in addition any supporting materials as part of the system. Hopefully the creation of

this system will both address the issues outlined above and be the start of many other

investigations to come.

Jayne Vidheecharoen

18

Works Cited

Bavinton, Nathaniel. “From Obstacle to Opportunity: Parkour, Leisure, and the

Reinterpretation of Constraints.” Annals of Leisure Research. 10.3. (2007): 391.

PDF File.

Crawford, Margaret. “Contesting the Public Realm: Struggles Over Public Space in

LA.” Journal of Architectural Education. 49.1. (1995): 4-9. PDF File.

Dennett, Andrea. Weird and Wonderful: The Dime Museum in America. NYU Press,

1997. Print.

Harris, Neil. Humbug: The Art of P. T. Barnum. Phoenix ed. University Of Chicago

Press, 1981. Print.

Hilbrecht, Margo. “Changing Perspectives on the Work–Leisure Relationship.”

Annals of Leisure Research. 10.3. (2007): 368-384. PDF File.

Huizinga, Johan. Homo Ludens: The Play Element in Culture. Boston, Massachusetts:

Roy Publishers, 1950. Print.

McGonigal, Jane. Reality is Broken: Why Games Make Us Better and How They Can

Change the World. New York, NY: Penguin Press, 2011. Print.

Jayne Vidheecharoen

19

Slavin, Kevin. “Kevin Slavin: How Algorithms Shape Our World.” Ted Talks,

Jul. 2011. Web. 4 Dec. 2011.

Stark, Kio. “Notes on Authenticity and Uncanny discussions.” For Reals: Fall 2010.

Oct. 2010. Web. 16 Jan. 2012.

Varnelis, Kazys. “Conclusion: The Rise of Network Culture.”

Networked Publics. Web. 4 Dec. 2011.

Wark, McKinzie. A Hacker Manifesto. Cambridge, Massachusetts: Harvard University

Press, 2004. Print.

Jayne Vidheecharoen

20