Prototypingin placep1519 bowser

Prototyping in PLACE: A Scalable Approach to Developing Location-Based Apps and Games

Anne Bowser University of Maryland College Park, MD USA

[email protected]

Derek Hansen, Jocelyn Raphael, Matthew Reid, Ryan Gamett

Brigham Young University Provo, UT USA

[email protected]

Yurong He, Dana Rotman, Jennifer Preece

University of Maryland College Park, MD USA [email protected], da-

[email protected], [email protected] ABSTRACT The rising popularity of location-based applications and games (LBAGs) that break spatial, temporal, and social boundaries creates new challenges for designers. This paper introduces PLACE, an iterative, mixed-fidelity approach to Prototyping Location, Activities, Collective experience, and Experience over time in LBAGs. PLACE consists of 6 design principles: start small and scale up the fidelity, treat participants as co-designers, test in a representative space, focus on activities more than interfaces, respect authentic social experience, and represent time authentically. The effectiveness of PLACE was evaluated by prototyping Floracaching, a geocaching game for citizen science. This revealed the types of insights that PLACE provides, best practices for implementing PLACE, and how PLACE com-pares to other prototyping methods.

Author Keywords Mixed-fidelity prototype; PLACE; location; mobile apps; location-based games; co-design

ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

INTRODUCTION The widespread adoption of location-enabled mobile phones is opening a new frontier for the designers of games and applications. Early location-based apps and games (LBAGs) that initially required specialty GPS equipment, such as Geocaching, are now accessible to millions of peo-ple with Smartphones (according to the latest PEW report, almost half of all American adults are Smartphone owners). New games and applications, such as Foursquare and MyTown, are defining a growing market; extant applica-tions, such as Facebook, are integrating location to increase their appeal and relevancy. LBAGs connect users with local businesses, facilitate social interaction, and even help col-lect useful information like geospatial data [20].

Existing techniques are insufficient for prototyping and evaluating these popular new applications and games. LBAGs break the spatial, temporal, and social boundaries that define traditional software prototypes. They allow users to play wherever they can carry a Smartphone, play while engaging in other activities, and adopt social roles that de-viate from their real-life identity [26].

Evaluation techniques like controlled usability tests fail to consider the holistic experience [5] of mobile games includ-ing social interaction, movement, and experience over time. Unfortunately, the vast majority of mobile research is still conducted in a controlled setting. According to one review, only 5 of 50 papers on mobile development and 8 of 38 papers on mobile evaluation reported on field research [16].

Developers acknowledge the unique challenges to prototyp-ing and evaluating LBAGs. Radio beacons, Bluetooth, and RFID tags [28] helped create early location-aware proto-types. For more advanced prototypes, systems like Con-textPhone [27] and MyExperience [11] allow developers to collect user experience data in the context of use. Still, the-se solutions are limited. Techniques that teach developers to represent location focus solely on that aspect, neglecting important factors like game activities, social experience, and experience over time. Solutions that consider context more holistically "collect context as a resource" but fail to teach developers which aspects of context are important or how to use context to improve their design [27].

With these limitations in mind we have constructed the PLACE approach (PLACE) for comprehensively prototyp-ing LBAGs using standard mobile phone functionality. PLACE is a scalable, mixed-fidelity approach to Prototyp-ing for Location, Activities, and Collective Experience over time. The acronym PLACE represents the crucial ele-ments involved in prototyping LBAGs, but PLACE is also a set of principles (see table 1).

The aim of this paper is twofold. First, we describe the PLACE approach and how it can be used to prototype LBAGs. Second, we use the development of Floracaching, a location-based game for citizen science, to evaluate PLACE with the following research questions in mind:

• What are the best practices for implementing PLACE?

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHI 2013, April 27–May 2, 2013, Paris, France. Copyright © 2013 ACM 978-1-4503-1899-0/13/04...$15.00.

Session: Game Design CHI 2013: Changing Perspectives, Paris, France

1519

• What design insights can (and cannot) be derived from the PLACE approach?

We begin by contextualizing PLACE in relation to other prototyping concepts and techniques, especially those ele-ments unique to LBAGs. We funnel this discussion into a presentation of the PLACE principles. After discussing our method including our case study of Floracaching, we con-clude by discussing the research questions posed above.

BACKGROUND Prototyping strategies allow designers to test and iteratively improve their initial designs. Prototypes range in their fidel-ity, or the accuracy of their representation to a finished product, and enable users to interact with the physical mani-festation of an idea. Low-fidelity prototypes such as paper prototypes are designed to quickly and inexpensively eval-uate alternate ideas [29]. High-fidelity, interactive proto-types enable users to evaluate advanced interfaces but are expensive and time consuming to build.

Recent research expands the discussion of prototypes be-yond the traditional high- vs. low-fidelity debate [32]. Mid-fidelity prototypes fall in the middle of the prototyping spectrum. Mixed-fidelity prototypes are high-fidelity in some aspects and low-fidelity in others [21]. For example, a prototype for a mobile multi-player game that displays a finalized interface but does not allow players to communi-cate with each other could be considered a mixed fidelity prototype. We describe PLACE as a mixed-fidelity proto-type because out of the four elements unique to prototyping LBAGs, any element can be high-or low fidelity.

The first element of PLACE is location. Without evaluating an app or game in its real-world context it is difficult to determine whether the events that a game depends on actu-ally happen, or to determine how these events play out [25]. One way to assess the impact of location is to “bodystorm,” or brainstorm in the context of use [26]. Other methods for in situ evaluation involve annotating mobile interfaces in the field [33] and using photographs, notes, and sketches to record in-game experience [3]. The objective of prototyping for location in PLACE is to allow players to experience a space that mimics the target environment.

The second element of PLACE is activities. Activity-centered design is a theoretical framework that understands computational activity as complex, collectively negotiated, and socially embedded [12]. Drawing on this paradigm, developers of ubiquitous computing applications suggest designing for activities as "first class objects" that are tested in the context of use [7]. As relatively new interaction gen-res, LBAGs pose many unresolved questions about how to structure activities and what to include in game play [19]. Including activities in the prototyping process helps design-ers evaluate and refine these core elements. Therefore, PLACE requires that designers treat in-game activities as any other feature of a prototype – as tentative, subject to user evaluation, and inviting iterative improvement.

The third element of PLACE is collective experience. Pro-totyping for collective experience requires testing with mul-tiple players or a representation of other players [25]. At the very least, a designer or automated program must adopt the role of a player when evaluating technologies with a strong social component. An interaction can be staged, with differ-ent actors collectively performing different scenarios [6]. Alternatively, social interaction can be evaluated by provid-ing a group of users with a prototype and telling them to use it socially [17]. With the understanding that most location-based games support collective experience (and many re-quire it), PLACE requires a prototype to be evaluated in social conditions similar to those in the finished game.

The final element of PLACE is experience unfolding over time. Because it requires time to move from one geographic location to another, all LBAGs have a temporal component [23, 31], and many social location-based games (e.g., Geo-caching) play out over more extended periods of time. Test-ing a prototype in situ can partially represent how a game plays out [33]. Testing different scenarios in the order that they occur with a Wizard of Oz prototype is another ap-proach to representing time [15]. Turn taking is a third way of condensing time, although this experience might not transfer to a game that, when finished, offers continuous play. PLACE suggests that the prototyping process should organize sessions so that they accurately reflect the way actual users will experience time in the app or game.

THE PLACE APPROACH PLACE is a generic prototyping approach covering the cru-cial elements of LBAGs and set of key principles rather than a specific toolkit or technology. Thus, the approach could be implemented with the help of existing toolkits and techniques. The key principles of PLACE (table 1) are:

PLACE Principle Elements

Start small and scale up the fidelity All

Treat participants as co-designers All

Test in a representative space Location

Focus on activities more than interfaces Activities

Respect authentic social experience Collective Experience

Represent time authentically Time

Table 1. The 6 PLACE Principles.

Start small and scale up fidelity Recognizing the importance of iterative design [22], we refer to PLACE as scalable because the method is practiced in multiple iterations where the fidelity of each aspect of the prototype is greater than or equal to its fidelity in the last iteration. For example, an early prototyping session may take place on one floor of a building (low-fidelity location), while a later prototyping session may take place on the building's grounds. PLACE is a mixed-fidelity approach in


1520

recognition that some aspects of a location-based prototype can be higher fidelity than others.

Treat participants as co-designers Participants in PLACE prototyping sessions should be treated as co-designers, not merely evaluators or disinter-ested parties. Indeed, participants should include design team members who play alongside those new to the game as participant observers. This approach, which borrows from [1, 8] allows designers to act as expert players, gain their own first-hand experience, and ask design-related questions. Meanwhile, the involvement of participants in-fuses new ideas and challenges existing assumptions.

An important element of co-design is to hold design-centric focus groups after each session so that players can critique the prototype, brainstorm improvements, and discuss how the LBAG would fit into their own lives and social experi-ence. Other methods to collect data from participants, in-cluding surveys/questionnaires, observation during play, and individual interviews may be added to assure that eve-ryone’s opinions are shared and to limit the effects of group-think, which can emerge in focus groups [30].

Test in a representative Place As discussed earlier, there is no perfect substitute for the physicality of moving around in space. PLACE recognizes that prototyping sessions of location-based games should always include location, even if the location dimension may be initially constrained to a smaller location than the final app or game will take place in. Furthermore, at least some of the sessions should be conducted in the exact physical context of the final LBAG. For example, if an app is de-signed to be used outdoors, at least some prototyping ses-sions should be conducted outdoors. Likewise for LBAGs intended for use in loud and crowded spaces.

Focus on activities more than interfaces PLACE considers prototyping for activities, user motiva-tions, and game dynamics more important than prototyping for interface design. Even if user interfaces are designed perfectly, activities that are not engaging will lead to sub-par apps and games. Thus, PLACE recommends using high fidelity activities (i.e., activities likely to be used in the final LBGA) even at early prototyping sessions. In contrast, low or medium fidelity interfaces are typically sufficient.

This is not to downplay the importance of getting the inter-face right, since interface problems can completely disrupt or preclude experiences that rely upon them [23]. However, designing increasingly complex interfaces is an aspect of prototyping that most developers are proficient in. Further-more, scaling up interfaces is not a unique challenge to the iterative design of LBAGs.

When testing activities, prototyping sessions should allow users some flexibility in what they choose to do, as their choices can be important indicators of what they value.

Figure 1. PLACE is an iterative, mixed-fidelity approach

that scales up the main elements of LBAGs- Location, Activities, and Collective Experience over time.

Respect authentic social experience PLACE is designed to prototype LBAGs that have a signif-icant social component. Because LBAGs often play out in local settings, players can use them to interact with existing friends, as well as strangers who share their locations. As such, if an LBAG has these characteristics, then PLACE prototyping sessions should be group exercises that have the same mix of existing and new relationships as the final app or game expects. If the final app will be used by groups of existing friends, or by family units, then the prototyping sessions should also. Furthermore, if the final LBAG will encourage users to invite friends to play mid-way through gameplay, so should the prototyping session. While this poses challenges for educating newcomers and collecting informed consent, tackling those hurdles allows for a more authentic social experience that can lead to insights about how the LBAG fits into users’ everyday social experiences.

Represent Time Authentically By design, prototyping sessions need to have a fixed dura-tion, unlike many LBAGs. However, the way in which time is dealt with should mirror the way time is dealt with in the final app as closely as possible. Though time will need to be condensed, prototyping sessions should allow for synchro-nous or asynchronous play, depending on the nature of the final app. For example, if the final app involves asynchro-nous, undirected activity (such as the free play that charac-terizes Geocaching), then prototyping sessions, whether lasting an hour or a week, should involve the same.

METHODS To evaluate the PLACE approach we conducted a field trial of Floracaching, a location-based game described below.

Floracaching Overview Floracaching is an example of a location-based serious game, and is described in depth elsewhere [3, 12]. It is re-lated to geocaching, but instead of physical caches, specific plants serve as the site of virtual caches. It also differs from geocaching in its aim of helping collect useful scientific


1521

data including plant phenology data, which can be used to study topics such as the effects of climate change and the dissemination of allergens [13]. A screen shot of the Flo-racaching interface is presented in the right side of Figure 4.

The game is designed to appeal to two primary populations: gamers interested in location-based games such as geocach-ing and Letterboxing, and citizen scientists interested in themes relating to botany, ecology, and climate change. In general, gamers are motivated by game mechanics such as challenge, theme, reward, and progress [10]. Players of location-based games are also motivated by a desire to ex-perience nature, socialize, and perform scavenger hunt ac-tivities [24]. In contrast, citizen scientists have different motivations based on personal interest in science as well as more altruistic wishes to facilitate scientific work [31].

Floracaching is an ideal LBAG to evaluate the PLACE ap-proach for several reasons. As an application with activities, location, social experience, and temporal play, Floracaching has many of the components exhibited by common loca-tion-based games. Furthermore, our team has complete con-trol over developing the application and needed input on a complete redesign for a future Floracaching iPhone app.

Study Design The PLACE approach was evaluated by a field study of its use in prototyping a future Floracaching app at two Univer-sities by different groups of facilitators. A total of 6 differ-ent design sessions were conducted, 3 at each site, as de-scribed in detail later. Using two sites generated independ-ent data from different geographic locations and different groups of people. It also allowed us to identify and test best practices for implementing PLACE, since we could take lessons learned from one location and apply them to the other. Note that the two sites were not used as different experimental conditions, which would be premature. In-stead, they were used to help understand the variability of results derived from different implementations of the PLACE approach.

Data Collection & Analysis A variety of data sources were collected at each of the 6 co-design sessions. Surveys and focus groups provided data on participants' perceptions. An analysis of the badges that players earned taught moderators which activities players completed. Notes taken by facilitators reflected on the suc-cesses and failures of each phase of PLACE.

The post-session survey contained 16 short questions ad-dressing Floracaching, the co-design experience, plant knowledge, Smartphone use, and demographics. Questions about Floracaching included questions about general per-ceptions of Floracaching ("How likely would you be to participate in Floracaching?"), game content ("Which was your favorite activity you performed today and why"), and motivation ("What would motivate you to participate in Floracaching or a similar activity"). Questions about the co-design experience asked participants which aspects of co-

design they enjoyed, and which aspects could be improved. The remaining questions addressed participant knowledge and demographics. 48 participants completed the survey.

The post-session focus groups allowed facilitators to collec-tively explore alternative activities, user motivations, and ideas for improving Floracaching and the prototyping pro-cess. Facilitators (including two who were responsible for programming the app) and participants were encouraged to participate, though there was one designated focus group moderator. As with survey questions, focus group questions covered both the Floracaching app and the co-design pro-cess. Sample focus group questions included "What ideas do you have for future activities?" and "How do you think people will use Floracaching in the 'real world'?"

Behavioral trace data captured during the sessions provided exact counts of how often various activities were attempted and completed. Spreadsheets were used to calculate the number of badges awarded, comments left on webpages, and caches checked into for each player. Additionally, after each of the 6 design sessions facilitators independently wrote down their impressions of what went well, what could have gone better, and suggestions for future sessions. Facilitators then met in de-brief meetings to discuss their write-ups and generate specific suggestions for future ses-sions. Facilitators also sent summary comments about the overall experience after all sessions were completed.

Data from players and facilitators was analyzed using a grounded theory approach, where the themes emerged from the data [8]. Raw data was organized into categories based on the 5 aspects of PLACE. Data from each category was aggregated into common themes in an iterative process.

As an example, the comment “You can maybe link up with people who are within 20 miles of you, and see what plants they're seeing” was categorized as a comment referring to location, social experience, and the co-design process. In the context of location, this comment might be understood as an awareness of how space functions in the game. In the context of social experience, this comment might be under-stood as a desire for indirect social interaction. In the con-text of co-design, this comment might reveal a willingness to make suggestions that improve the game.

PROTOTYPING FLORACACHING WITH PLACE This section describes how we implemented the PLACE principles (shown in italics) in our Floracaching prototypes. We also address our first research question by identifying best practices for implementing PLACE based on our suc-cesses and failures.

Overview In order to start small and scale up the fidelity, we evaluat-ed 2 iterations of Floracaching, which are referred to as phase I and phase II. Both phases were designed to repre-sent all of the key aspects of PLACE. Phase I was a mixed fidelity prototype, with each of the aspects (except activi-ties) being low- or mid-fidelity. In scaling up PLACE for


1522

phase II, we increased the fidelity of some aspects and kept the fidelity of other aspects the same (see Figure 2).

A total of 6 independent group sessions were split evenly across two different universities. Plant experts (proxies for citizen scientists) and technology and gaming enthusiasts (proxies for gamers) were recruited separately and split into separate phase I sessions so that those without certain skills (i.e., plant knowledge or familiarity with QR-code readers) would not feel overwhelmed by others who were clearly experts. This is a best practice in focus group design for similar reasons [29]. Participants were recruited from uni-versity classes and the personal networks of the design team. In order to respect authentic social experience, partic-ipants were encouraged to bring their friends and family to design sessions. Participants were not paid, although some received extra credit. Table 2 shows information on the participants of each session.

Session Participants

(including de-sign team)

Demographics

University A Phase 1 7 (2) Male: 1, Female: 6

Plant Experts: 7

University A Phase 1

7 Male: 7, Female: 0 Plant Experts: 0

University B Phase 1

9 (1) Male: 2, Female: 7 Plant Experts: 8


11 (2) Male: 4, Female: 7 Plant Experts: 0

University A Phase 2

14* (2) Male: 9, Female: 5 Plant Experts: 2


10* (2) Male: 1, Female: 9 Plant Experts: 5

All sessions 58 Male: 24, Female: 34 Plant Experts: 22

*This number includes registered users who actively participated. Family and friends were with some of them.

Table 2. Summary of sessions and participants

Phase I: Building-wide, 1.5 hour lower fidelity session Prior to running the sessions, facilitators placed 35-40 plant mounts (see left-hand image in Figure 3) or photographs throughout multiple floors of a building. These served as caches. In order to test in a representative space, caches were clustered into groups that represented forests. Placing caches on different floors represented geographic distance between plants.

Each cache was attached to a different QR code, which pointed to a unique page on a Wordpress site. The site showed a photograph of the plant cache and allowed users to “check in” by leaving a comment using their mobile phone. Facilitators represented the remaining game inter-face with a paper packet that included a map of selected

caches, a description of un-mapped caches, a dichotomous key to aid in classification, and a list of activities.

The activities chosen for the prototype were the same activ-ities that were being considered for the Floracaching app. They fell into 4 broad categories, as shown in Table 3, though these categories were not revealed to users. This approach allowed us to focus on activities more than inter-faces. Badges, in the form of sticky-notes with hand-drawn images (see left-hand image in Figure 4), were created for each activity to help motivate and reward participants.

Figure 2. Phase I (left) and phase II (right) diagrams

showing fidelity of the 4 key aspects of PLACE as used in the Floracaching study.

Figure 3. Caches from phase I (left) and phase II (right).

Figure 4. The tourist badge from phase I (left) and

screenshot with badge from phase II (right).


1523

Activity type Purpose Sample activities

Scavenger Hunt

Find objects in a category

Coniferous Collector, Forb Finder, Order of the Oak, Invasive Patroller

Social Interact with other players

Friendly Floracacher, Tour Guide, Tourist

Creation/ Validation

Add new caches/ verify identification

First Finder, Validator

Citizen science

Contribute scientific data

Nature Conservationist, Budding Scientist

Table 3. Summary of Floracaching activities

Phase I sessions began with a 10-15 minute orientation where the facilitators introduced the idea of Floracaching, the participants’ role as co-designers (in order to treat par-ticipants as co-designers), and an outline of what to expect during the session. Facilitators also ensured that each par-ticipant knew how to use the technology (a Smartphone with a QR-code reader and browser) and paper materials.

After orientation, each participant was given a specific ac-tivity to complete as an introduction to the game. Once the sample activity was completed, participants were released for 30 minutes of free play to complete any activities they desired. This pattern of play represents time authentically, since the final Floracaching app will allow people to com-plete activities of their choice on their own time. They were also allowed to travel together, which happened often, in order to respect authentic social experience. Upon comple-tion of each activity, or at the end of the entire session, par-ticipants visited a facilitator to receive a badge (Figure 4). After free play was over, players gathered to turn in their badges, complete an individual survey, and participate in a 20-minute focus group. Sessions lasted about 90 minutes.

Phase II: Campus-wide, 2-3 week higher fidelity session Prior to launching the Phase II sessions, facilitators created 35-50 caches across a university campus. Caches were identified by placing garden stakes with a unique QR code next to specific plants (see right-hand image of Figure 3). This allowed us to test in a representative space, by not only positioning caches in different locations but in a natu-ral setting similar to the target setting of the app.

As in phase I, the user interface still relied upon a QR code reader and web browser, but in phase II the paper materials were replaced by content shared on an updated version of the Wordpress site. For example, the paper map from phase I was replaced with a Google map, the description of caches and activities were hosted on different mobile-friendly web pages, and the dichotomous key was replaced by links to the EOL page for each plant species. Participants still checked in to caches by leaving comments on the page as-sociated with the cache’s QR code. Facilitators, who used a Wizard of Oz technique [29], monitored new comments on

the website during the two weeks and added virtual badges to user profile pages (see right-hand side of Figure 4) after players completed activities. Though the interface was me-dium-fidelity, there was still a focus on activities more than interfaces.

One key difference between the phases was that phase II was played over an extended period of 2-3 weeks. This al-lowed people to play at their leisure in an unstructured, asynchronous manner that fit into their daily routines. The ability to begin playing after others have started, stop play-ing when interest wanes, and return to the game when inter-est is revived helped represented time authentically.

Phase II began with the invitation to attend an optional ori-entation session. Participants who participated in phase I did not need to attend, since they already knew the relevant background and understood their role as co-designer. Addi-tionally, the Wordpress site was designed with detailed in-structions and help documents so that people could join the session partway through and still be able to understand what was going on. This happened when some players in-vited their friends to Floracache with them. It also occurred several times when curious university students scanned a QR code and were prompted to email the facilitators to join the fun. These both helped to respect authentic social expe-rience, wherein players join the game partway through and are able to play alone or with friends.

Players who attended the orientation session were intro-duced to the technology and materials and given a sample activity to complete. Players who could not attend were sent a welcome email that explained their role as co-designer and referred them to the Wordpress website. Following the game, participants were emailed an individual on-line sur-vey to complete and invited to attend a focus group. Behav-ioral traces, such as comments left and badges earner, were captured by facilitators.

Best Practices for Implementing PLACE Below is a description of some best practices for imple-menting PLACE based on our successes and failures. They are organized around the 6 PLACE Principles.

Start small and scale up fidelity • Work on lower fidelity prototypes should be reused in

higher fidelity prototypes. Our inclusion of the same activities in both prototypes worked well, while our use of different tree species required significant extra work.

• If possible, hold multiple sessions for each phase. This assures that findings are robust across different groups and locations and allows the findings of one session to inform the way the next is conducted.

Treat participants as co-designers • Make changes to prototypes based on initial feedback

to show that you are listening. As discussed earlier, this approach led to an improved phase II prototype.


1524

• Have the design team participate alongside participants during gameplay and in focus groups, while assuring that nobody’s opinion is more valued than others. This was achieved through positive reinforcement.

Test in a representative space • Conduct early sessions in a constrained location (e.g.,

building) even if it is not the target setting. Our phase I building-level sessions focused the short sessions on activities, not confounding contextual factors.

• Conduct higher fidelity sessions in the target environ-ment. Our phase II (campus-wide) sessions allowed us to learn how Floracaching fit into people’s everyday lives and their reaction to working with live trees.

Focus on activities more than interfaces • Don’t test too many activities at once. In hindsight we

could have run multiple sessions with fewer activities to derive a final set. Labeling the difficulty of each ac-tivity also helps players choose what to focus on.

• Make sure interfaces meet minimum usability require-ments. Though the focus is on activities, a poor usabil-ity experience can seriously disrupt the prototyping sessions. For example, we realized that some QR code readers opened the Wordpress site in their own brows-er, which didn’t remember users who had logged in with the smartphone’s built-in browser.

Respect authentic social experience • Allow participants to recruit friends, family, and col-

leagues. This helps increase participation, keep people engaged, and facilitate natural social interactions.

• Allow people to join higher fidelity prototypes mid-way through. This is how actual apps are disseminated. Better marketing would have helped recruitment through the duration of phase II.

Represent time authentically • Collect data at regular intervals of time for high fidelity

sessions that last multiple weeks. Doing so allows for modifications, keeps participants engaged, and solicits their input while their experiences are fresh.

• Plan for a flexible timeframe for high fidelity sessions. Both of our high fidelity sessions were extended in or-der to allow new participants to play, requiring us to contact the university grounds crew for permission.

FLORACACHING FIELD TRIAL RESULTS This section answers our research question: What design insights can (and cannot) be derived from the PLACE ap-proach? We report what we learned prototyping Floracach-ing, mapping each set of findings to a principle of PLACE.

Start small and scale up the fidelity The process of scaling up a prototype is an essential aspect of PLACE. As PLACE is an iterative process, the insights from our first prototype informed the design of our second. For example, players in phase I disliked some aspects of our low-fidelity interface like "[using] the paper to come up

with plant names- there was a lot of flipping back and forth." When we moved into phase II we were able to use this feedback to improve the usability of the app by using hyperlinks to ensure a more intuitive information flow. Similarly, phase I participants wanted to see a list of all trees in a particular tour when visited the first tree in a tour, which we were able to implement in our phase II prototype. Learning these and other insights in our relatively low-cost phase I sessions allowed us to solve problems prior to the relatively costly phase II sessions, which could then provide new insights related to the higher fidelity prototype.

We also gained important insights from comparing how participants interacted with the phase I and phase II proto-types. "Forb finder" was a very unpopular activity, with only 5 out of 48 players listing it in their top 3 activities. During phase I we wondered whether that might be because participants disliked scavenger hunt activities in general. In University B's phase II session, we replaced Forb finder with Order of the Oak due to differences in biodiversity. Order of the Oak was one of the most popular activities, with 5 participants earning the badge. Therefore, comparing phase I results with phase II results taught us that it was forbs, not scavenger hunt activities in general, that players disliked. By scaling up PLACE designers can disambiguate potentially confusing results.

Treat participants as co-designers Floracaching players offered two types of high-level evalu-ations. First, players extrapolated on their personal experi-ence to explore what “most people” or “other types of peo-ple” might like. Gamers hypothesized about the desires of plant experts; plant experts noted that outside of their user group “you might need more of a reward system or some-thing to get people to actually use it.”

Second, beyond expressing what they liked and disliked, players offered concrete suggestions for how to improve the game. Gamers suggested “a Wiki associated with each plant,” augmented reality with social tagging, and mecha-nisms for better supporting tours. Plant experts discussed ways to address the challenges of ensuring data quality and scientific rigor. Though not all suggestions were reasona-ble, some suggestions were incredibly valuable; for exam-ple, a participant’s idea of starting with a simple activity and then introducing more complex activities over time is being built into the current application.

Furthermore, it was clear that participants enjoyed the co-design process. When asked, "What did you like best about today's co-design experience?" some participants comment-ed on the nature of interaction with facilitators, mentioning "The friendliness of the graduate student researchers" or "Meeting [the facilitator] and learning more about her work." Treating participants as co-designers can help de-signers ensure that their application appeals to diverse audi-ences, generate concrete suggestions on how to improve a design, and create a more enjoyable experience.


1525

Test in a representative space For both sessions Floracaches were dispersed over a bound-ed geographic space. Some Floracaches were clustered to-gether; others required that players travel longer distances to reach them. This experience of moving through space created a sense of place [14] that led players to contemplate the role of location in Floracaching.

One player, whose knowledge of mobile games and appli-cations exceeded her knowledge of plants, wanted to Flo-racache “in a place that I was familiar with. There are 3 or 4 wild places close to me that I go to... I hate getting lost.” Other players want to use Floracaching to explore their en-vironment, “to see what's available, what's around here,” or to take tours that others found interesting. Our phase II study also showed how players’ integrated space into their everyday lives, such as Floracaching on their way to class. Knowing how players perceive location and how it fits with their lives supports a more informed design. For example, our insights suggest the need to develop features that high-light nearby caches at a given moment.

Testing in a representative place also reveals the challenges associated with finding and checking into a specific loca-tion. While a Google map was specific enough to lead play-ers to large trees, the granularity was insufficient to point to small flowers and shrubs. This understanding, and ideas for overcoming it (such as showing photos of a tree in context or describing a plant relative to other landmarks) will be essential in our final GPS-based design. Testing our proto-type outside also taught us that we need a backup plan to allow users to check into caches if the technology fails them (e.g., the GPS isn’t accurate). We learned this from hearing frustrated comments of phase II participants who couldn’t check into a cache because of a missing QR code stake.

Activities are more important than interfaces Using a mixed-fidelity prototype helped players think holis-tically about the game experience. Our creation of a geo-caching game for citizen science was validated early in the design process; in the words of a phase I participant, “With Geocaching it's cool, and it’s fun, but it's like 'what's the point' whereas for this, you're contributing to science while you're doing it.” Another participant thanked the facilitators for inviting her to be a part of such a fun activity.

The focus on activities also helped us identify which activi-ties to include in the finalized app. Participants honed in on a few activities such as checking in with a friend (the most frequently completed and highest rated in surveys), partici-pating in scientifically beneficial activities (the second most popularly chosen activity; "It was good to be contributing as well as enjoying the activity"), validating caches (the third most popularly chosen activity; "It's cool to go back in and check someone's work"), and taking tours ("It was fun to just enjoy something that someone else had made"). The-se insights proved enormously valuable when designing the final app by pointing to essential activities to include.

We also learned that focusing on activities does not prevent users from evaluating an interface. Our players gave us unsolicited feedback on the clarity of links, the need for plant images on the map, and the location of badges. Priori-tizing activities over interface may give developers insights on activities and motivations that they might not otherwise realize, but insights into user interfaces are also available.

Represent authentic social experience Location-based apps and games afford, but do not mandate, social participation. Some players considered Floracaching strictly an individual activity; according to one, “If I did this I would definitely want to do it alone... it would just be internalizing, a happy nature state.” Others felt differently: “If I did Floracaching in the future, I would enjoy doing it with others.” In fact, the most desirable (based on survey) and most commonly completed activity was Friendly Flo-racacher (checking into a cache with a friend). Still others would Floracache individually or with friends depending on which activities they were trying to complete.

The ability to see other player's comments led one player to characterize Floracaching as “a social activity that you can actually do alone.” One player “tried to leave just funny comments, so that if someone else was reading them they could hopefully get a little bit of entertainment.” Echoing the sentiment of “a social activity that you can do alone,” another player thought that the game should add a feature allowing players to “link up with people who are within 20 miles of you, and see what plants they're seeing 20 miles away.” We now know that individual activities, direct so-cial activities, and indirect social activities need to be sup-ported in the finished application. By implementing PLACE in a way that enables the types of social interaction permit-ted in the finalized game, developers learn which types of social interaction players will actually practice.

Represent time authentically Like many location-based games, Floracaching is designed to be played in the fleeting moments of spare time between more structured activities, or during another planned activi-ty (e.g., a family hike). Incorporating “free play” into the prototyping sessions revealed how players might use Flo-racaching in their real lives. Some players did use Flo-racaching while transitioning from one activity to another, “especially walking between classes: I could just snap a couple of them, and keep going.” 56% of the 181 phase II check-ins occurred during weekday, daytime hours when most classes are held. The other 44% occurred during weekend evenings (26) or weekends (55), often as part of a burst of check-in activity (e.g., over 10 check-ins that day). These visits showed deliberate planning by participants Floracaching in order to explore campus or teach others about plants. We now know that both planned expeditions and spontaneous play should be supported by the app.

Players realized that they were unlikely to Floracache every day and evoked longer time frames when discussing the app. One player thought it would be interesting to see “if


1526

plants come back the next year, and when they bloom.” Another player wanted to be reminded of Floracaching: “If there were push notifications I might have walked around campus the other day and thought ‘really, I'm within 20 feet of something?’” If Floracaching is played intermittently and over time, a mechanism should exist to remind players of things they might forget such as the Floracaches they have visited and the badges they are working towards. The pro-cess of representing time in PLACE can teach developers whether they need to offer game mechanics that supplement human memory or otherwise support use over time.

DISCUSSION Rather than a specific prototyping technique, PLACE is an approach to prototyping that can rely upon many tech-niques. While prototyping Floracaching we used elements of techniques such as prototyping in context [4, 31], Wizard of Oz [15], design-centered focus groups [9], and contextu-al inquiry [2]. Other implementations of PLACE could in-corporate more sophisticated techniques for prototyping location, such as the use of beacons, or other location proto-typing toolkits [10, 25]. One technique that PLACE does not incorporate is traditionally usability testing. After PLACE takes a designer through the early and middle stag-es of developing a game or application, traditional usability testing will be necessary to formally assess the interface.

PLACE is not the first framework that remixes existing prototyping techniques for use in a unique domain. The ActivityDesigner system allows developers of ubicomp applications to consider real-life activities and realistic time spans during in situ prototyping [18]. Trajectories is a con-ceptual framework that urges designers of complex inter-faces (such as those used augmented reality) to consider how users move through space, time, activities, and inter-faces [1]. PLACE shares similarities and differences with both. For example, both PLACE and ActivityDesigner are iterative approaches that consider activities, time, and loca-tion. PLACE also emphasizes collective experience, a fac-tor essential to LBAGs, and the need to scale up all the di-mensions of a prototype. Our contribution is that PLACE is the first framework designed specifically for the use of building LBAGs and the first framework that addresses all of the complex factors unique to the design of LBAGs.

We anticipate that PLACE could be used to prototype many types of LBAGs including ones similar to Facebook places, Foursquare, and other “check in” based apps. We also rec-ognize that the unique characteristics of an app will deter-mine which aspects of PLACE to emphasize at which stag-es in the design process. Because we hypothesized that the gamers who use Floracaching might prefer different activi-ties than the plant enthusiasts who use Floracaching it was important for us to test activities at a high fidelity even in early iterations of PLACE. In contrast, developers of a competitive LBAG might find it more important to consist-ently evaluate social experience at a high fidelity; develop-

ers of an augmented reality game might consider it crucial to develop a high-fidelity interface early on.

PLACE was evaluated while using a co-design strategy to prototyping Floracaching at two different locations with two groups of facilitators. Facilitators at both locations not-ed that the co-design strategy, which encouraged partici-pants to critically and constructively evaluate both the Flo-racaching app and the co-design process, was tremendously effective at generating insights that improved the design. However, other researchers have found that participants may hesitate to criticize a design when presented with a single alternative in traditional usability testing [34]. On one hand, this issue may be compounded by the comradery expressed between co-designers and facilitators. On the other hand the co-design process, which is designed to level the power imbalance of traditional usability testing, may empower participants to take a more critical stance. De-signers who utilize PLACE and other co-design methodol-ogies would be wise to keep this tension in mind.

PLACE is an iterative mixed-fidelity prototyping strategy developed to address the complexity inherent in building location-based games and apps that break social, spatial, and temporal boundaries. As such, PLACE encourages de-signers of LBAGs to consider Interface, Location, Activi-ties, Collective experience, and Experience over time early in the design process. While implementing PLACE requires significantly more time and effort than techniques such as sketching and Wizard of Oz, our field trial prototyping Flo-racaching suggests that the value of the insights gained from prototyping with PLACE is sufficient to merit the extra work involved. Using PLACE ensures that designers will know the activities their design needs to sustain, the places their design will be used in, the types of interactions their design should support, and how the use cases for their design will evolve over time. Utilizing this knowledge can lead to better designs for complex LBAGs.

ACKNOWLEDGEMENTS We would like to thank our co-design participants. This work was supported by NSF grant #CNS-0627084.

REFERENCES 1. Benford, S., Giannachi, G., Koleva, B. and Rodden, T.

From interaction to trajectories: Designing coherent journeys through user experience. Proc. Of CHI 2009, ACM Press (2009), 709-718.

2. Beyer, H., and Holtzblatt K. Contextual Design. Mor-gan Kaufmann Publishers, San Francisco, CA, USA, 1998.

3. Bidwell, N. and Holdsworth, J. Battleship by foot: Learning by designing a mixed reality game. In Proc. IE 2006, ACM Press (2006), 1107-1116.

4. Bowser, A., Hansen, D., Rotman, D. and Preece, J. Low fidelity prototyping for social, location-based games. Paper presented at the Mixed Reality Games Workshop, CSCW 2012, Seattle, WA.


1527

5. Buchneau, M. and Fulton Suri, J. Experience prototyp-ing. Proc. DIS 2000, ACM Press (2000), 424-433.

6. Burns, C., Dishman, E., Verplank, W. and Lassiter, B. Actors, hairdos & videotape – Informance design. Proc. CHI 1994, ACM Press (2004), 119- 120.

7. Christensen, H. and Bardram, J. Supporting human activities- exploring activity-centered computing. Proc. UBICOMP 2002, ACM Press (2002), 107-115.

8. Corbin, J. & Strauss, E. (2007). Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage Publications, Thousand Oaks, CA, 2007.

9. Druin, A. The Role of Children in the Design of New Technology. Behaviour and Information Technology, 21, 2 (2002), 1-25.

10. Flatla, D., Gutwin, C., Nacke, L., Bateman, S., and Mandryk, R. Calibration games: Making calibration tasks enjoyable by adding motivating game elements. Proc. UIST 2011, ACM Press (2011), 403-412.

11. Froehlich, J., Chen, M., Consolvo, S., Harrison, B. and Landay, J. MyExperience: A system for in situ tracing and capturing of user feedback on mobile phones. Proc. MobiSys 2007. ACM Press (2007), 57-70.

12. Gifford, B. and Enyedy, N. Activity centered design: towards a theoretical framework for CSCL. Proc. CSCL 1999, ACM Press (1999), 189-196.

13. Graham, E., Vassello, D., Gerrick, S., Han, K., and Hsieh, N. Challenges of mobile phone-based, GPS-dependent gaming for citizen science. Paper presented at the Mixed Reality Games Workshop, CSCW 2012, Seattle, WA.

14. Harrison, S. and Dourish, P. Re-place-ing space: the roles of place and space in collaborative systems. Proc. CSCW 1996, ACM Press (1996), 67-76.

15. Höysniemi, J., Hämäläinen, P. and Turkki, L. Wizard of Oz Prototyping of Computer Vision Based Action Games for Children. Proc. IDC 2004, ACM Press (2004), 27-34.

16. Kjeldskov, J. and Graham, C. A review of Mobile HCI Research Methods. Proc. Mobile HCI 2003, Springer-Verlag (2003), 317-3352.

17. Kurvinen, E., Koskinen, I. and Battarbee, K. Prototyp-ing social interaction. Design Issues, 24, 3 (2008), 46-57.

18. Li, Y. and Landay, J. Activity-based prototyping of ubicomp applications for long-lived, everyday human activities. Proc. CHI 2008. ACM Press (2008), 1303-1312.

19. Lund, K., Lochrie, M., and Coulton, P. Enabling emergent behaviour in location based games. Proc. MindTrek 2010. ACM Press (2010), 78-85.

20. Matyas, S., Matyas, C., Schlieder, C., Kiefer, P., Mi-tari, H., and Kamata, M. Designing location-based mo-

bile games with a purpose- Collecting geospatial data with CityExplorer. Proc. ACE 2008, ACM Press (2008), 244-247.

21. McCurdy, M., Connors, C., Pyrzak, G., Kanefsky, B. and Vera, A. Breaking the fidelity barrier: An exami-nation of our current characterization of prototypes and an example of mixed-fidelity success. Proc. of CHI 2006, ACM Press (2006), 1233-1242.

22. Nielsen, J. Iterative user interface design. IEEE Com-puting, 26, 11 (1993), 32-41.

23. Norman, D. The Design of Everyday Things. Basic Books: New York, NY, USA, 1988.

24. O'Hara, K. Understanding geocaching practices and motivations. Proc. CHI 2008, ACM Press (2008), 1177-1186.

25. Ollila, E., Suomela, R. and Holopainen, . Using proto-types in early pervasive game development. ACM Computers and Entertainment, 6, 2 (2008), 1-17.

26. Oulasvirta, A., Kurvinen, E., and Kankainen, T. Un-derstanding contexts by being there: Case studies in bodystorming. Personal and Ubiquitous Computing, 7, 2 (2003), 125-134.

27. Raento, M., Oulasvirta, A., Petit. R.. and Toivonen, H. ContextPhone: A prototyping platform for context-aware mobile applications. IEEE Pervasive Computing 4, 2 (2005), 51–59.

28. Rashid, O., Bamford, W., Coulton, P., Edwards, R. and Scheible, J. PAC-LAN: Mixed-reality gaming with RFID enabled mobile phones. ACM Computers in En-tertainment, 4, 4 (2006), Article 3A.

29. Rogers, Y., Sharp, H. and Preece, J. Interaction De-sign: Beyond Human-Computer Interaction. Wiley Press, New York, NY, USA, 2011.

30. Rosenbaum, S., Cockton G., Coyne, K., Muller, M. and Rauch, T. Panel on Focus Groups in HCI: Wealth of Information or Waste of Resources? Proc. CHI 2002, ACM Press (2002), 702-703.

31. Rotman, D., Preece, J., Hammock, J., Procita, K., Han-sen, D., Parr, C., Lewis, D. and Jacobs, D. Dynamic changes in motivation in collaborative ecological citi-zen science projects. Proc. CSCW 2012, ACM Press (2012), 217-226.

32. Rudd, J., Stern, K. and Isensee, S. Low vs. high-fidelity prototyping debate. Interactions, 3, 1 (1996), 76-85.

33. de Sá, M. and Carriço, L Lessons from early stages of design of mobile apps. Proc. MobileHCI 2008, ACM Press (2008), 127-136.

34. Tohidi, M., Bouxton, W., Baecker, R. and Sellen, A. Getting the right design and the design right: Testing many is better than one. Proc. CHI 2006, ACM Press (2006), 1243-1252


1528

Prototypingin placep1519 bowser

Technology

Transcript of Prototypingin placep1519 bowser