Evaluating Digital Technologies

Elizabeth Bolander Meghan Stockdale

Research Questions

• What do visitors think about Gallery One and ArtLens?

• How does Gallery One and ArtLens affect the museum visit experience?

Process

• Literature review conducted

• Iterative process throughout the planning and development

– User testing (usability, content, tone)

– Audience panels

• Ultimately realized we had many different needs that required different approaches

Mixed Methods Approach

• Usability tests of all interactives

– 30 tests per lens, Collection Wall, and ArtLens

• Observations

– Timing and tracking maps recorded for 200 visitors

• Skype conversations with off-site users of ArtLens

Mixed Methods Approach

• Comprehensive in-depth analysis

– Visitor groups placed into one of four test groups

– Pre- and post-visit interviews

• Range of topics explored

– Visit videotaped with flip cameras

• Usually about 2 hours of video was recorded per visit

– Phone interviews completed 2-3 months later

• Testing re-use, memory, perception changes, etc.

Mixed Methods Approach

• User data from Lens stations and ArtLens tracked via Google Analytics

– More difficult to track the Lenses than ArtLens

• Very valuable counterpoint to qualitative data

Research in Action

• Visitor wayfinding concerns

Research in Action

• App labels on walls

Research in Action

• Off-site versus on-site needs

– On-site has the benefit of the actual space

– Off-site users looking for additional contextual information

Strengths and Challenges

• Significant amount of deep, nuanced data

– Learning more than just the impact of technologies, but a comprehensive look at the entire experience

• Data tracking and management can be difficult

• Analysis of overall project and preparing reports for a diverse array of internal teams with different priorities

Let’s Chat!

• What technologies do your museums use? How do you evaluate them?

• What challenges have you experienced? How did you handle them?

• What can we do together to create a community of practice around these types of projects?