Evaluation of human-centred design in CityVerve...4. Evaluation of human-centred design in CityVerve...

Learnings for large-scale demonstrators

Evaluation of human-centred design in CityVerve

Drew Hemment, Sebastian Weise, Feimatta Conteh

1

FutureEverything ReportOctober 2018

Authors

Drew Hemment University of Edinburgh & FutureEverything

Sebastian Weise PlaceChangers

Feimatta Conteh FutureEverything

Learnings for large-scale demonstrators

Evaluation of human-centred design in CityVerve

2

ContentsForeword 3

1. Executive Summary 4

2. A human-centred Internet of Things and Smart City2.1 About the CityVerve demonstrator2.2 Scope of this report and constraints to the evaluation

677

3. Human centred design in CityVerve3.1 Human-centred design methods and approach3.2 Phasing of human-centred design in CityVerve3.3 A closer look: community forums and targeted design interventions

3.3.1 Community forums – building literacy and defining community goals3.3.2 Targeted human-centred design interventions – delivering solutions based on citizens’ needs and concerns

9912141416

4. Evaluation of human-centred design in CityVerve4.1 Evaluation methods

4.1.1 Materials considered4.1.2 Analysis methods4.1.3 Data reporting

1818181819

5. Outcomes of human-centred design interventions5.1 BeeActive – Encouraging an active lifestyle5.2 See.Sense – Improving the city experience for cyclists5.3 Buzzin’ app – Discovering the city around us

20202223

6. Insights gained - lessons learned 25

7. A note for future large-scale demonstrators 33

8. Limitations to the evaluation 35

9. Final notes and next steps 36

10. List of references 38

Appendix: Interview guide 40

Appendix: Community Champion Workshop Planning Template 41

Appendix: Comments from event attendees 44

3

In CityVerve, we had a bold ambition. The partners in the project came together around the vision to put people at the centre of the Internet of Things (IoT) and the smart city. The language and rhetoric of the citizen-centred smart city has now entered the mainstream, and is commonly recited in a majority of smart city programmes in Europe. This orientation and these values are furthermore already commonplace in urban development and city service design. But the jury is still out on how to best involve citizens in the development of IoT application for cities. The hurdle remains of turning rhetoric into reality.

When writing the proposal for CityVerve, we made the case for a human-centred, civic minded approach, and also to involve art and artists centrally in the project. This was supported by the consortium partners, and the funder, InnovateUK. The result is that CityVerve had art and culture as one of its four themes, alongside more conventional themes for city demonstrators: transport, health and energy. It also had human-centred design and citizen engagement at the centre of the project narrative, as cross-cutting, and delivered by FutureEverything across the four project themes.

A movement advocating for, and demonstrating, an alternative to top-down city and technology design arose over the past decade amongst arts-based organisations and prominent thinkers – arguing for a ‘civics’ based approach to the smart city1. In a way unique to Manchester, CityVerve brought together one of the chief advocates of this bottom up approach, FutureEverything, with major technology vendors, such as Cisco, Siemens and BT, alongside the city authority, universities, and innovative SMEs such as Sparta Digital. In CityVerve, the history and values of a grassroots movement came to the forefront in an attempt to engage citizens in at the heart of IoT technology development for the city of Manchester.

Here we report on a number of the human-centred design interventions delivered in CityVerve, and present the case for a human-centred, civic-minded approach to IoT and smart city demonstrators. Our specific ambition in CityVerve was to introduce principles and methods we had used successfully at a smaller scale, and in cultural and community projects, to a large scale, industrial demonstrator. CityVerve’s aim has been to provide a blueprint for the smart city, and we hope that these efforts can inform and signpost wider adoption of human-centred and citizen-led principles and methods in the industrial sector.

Dr. Drew Hemment CityVerve Theme Lead

Foreword

(1) Hemment, Drew, & Townsend, Anthony (Eds.). (2013). Smart Citizens. FutureEverything.

4

1. Executive summary

In this evaluative report, we review the implementation of human-centred design in the CityVerve demonstrator, a large-scale collaborative effort to deliver cutting-edge IoT and smart city services for the city of Manchester together with citizens2. The report provides an overview of the design approach, and learnings for future IoT and smart city demonstrators. In this evaluative report, we review the implementation of human-centred design in the CityVerve demonstrator, a large-scale collaborative effort to deliver cutting-edge IoT and smart city services for the city of Manchester together with citizens. The report provides an overview of the design approach, and learnings for future IoT and smart city demonstrators.

The CityVerve project was a nationally-significant £10m demonstrator project aiming to provide an integrative test bed for future IoT applications along the area around Oxford Road in Manchester, known as The Manchester Corridor. One distinctive aspect of CityVerve was an attempt to open up the technology development process to citizens, and make it accessible to different audiences. The aim here was to enable the IoT services to better address the real needs of people, and to enable citizens to play a part of setting the direction of future IoT services. This was through a number of methods and activities:

i. Community KPIs: Goals and metrics developed with communities for each of the core themes of CityVerve.

ii. Community forums: A series of open public events, staged at relevant community venues, for people to engage with project themes and share interests and concerns about technologies being developed.

iii. Targeted design interventions: Human-centred design activities tailored to use case needs and engaging a target user group.

iv. Public pilots and demos: A series of public deployments of technologies developed in CityVerve to generate exploration and feedback.

v. Artworks and experiences: A programme of public art commissions creatively engaging with themes and issues in CityVerve

(2) The approach taken in CityVerve was outlined in a blog post in August 2016: Hemment, Drew. (2016). How to design a useful Internet of Things – Introducing human centred design to CityVerve. FutureEverything, Manchester. Retrieved from http://futureeverything.org/news/human-centred-design-cityverve

http://futureeverything.org/news/human-centred-design-cityverve

5

vi. Community Champions: Individuals specifically trained in creative facilitation methods with the core mission to reach out to relevant community groups, primarily through community forums.

For the report, we conducted interviews with project participants, and reviewed a large set of documents, including formative and outcome reports from targeted user research and community forum activities. The focus of this report is the community forums and targeted design interventions, as well as wider insights that can be drawn from the programme overall.

The report presents eight recommendations or signposts for future IoT and smart city demonstrators:

1. Establish community goals and challenges

2. Build advocacy for the benefits of a citizen-centred approach

3. Give time to creativity

4. Create contact zones with communities

5. Remain open ended to accommodate emerging requirements

6. Place emphasis on community building over market research

7. Manage expectations and give space to critical voices

8. Be persistent and the outcomes will follow

Together, these lessons provide insights the authors hope will be of value to anyone incorporating human-centred design and citizen engagement in any future demonstrator projects in the domain of IoT and integrated urban services. They furthermore make a strong case for doing so.

The report concludes that when increasing numbers of entities in the city can become interactive – sending and digesting data generated through the interactions of people in the public domain – it is paramount to adopt effective approaches to build literacy, relevance and agency for citizens, and to design services that are useful, usable, and likely to be used.

6

(3) Hemment, Drew, & Townsend, Anthony (Eds.). (2013). Smart Citizens. FutureEverything.

(4) The approach taken in CityVerve was outlined in a blog post in August 2016: Hemment, Drew. (2016). How to design a useful Internet of Things – Introducing human centred design to CityVerve. FutureEverything, Manchester. Retrieved from http://futureeverything.org/news/human-centred-design-cityverve

2. A human-centred Internet of Things and Smart City

Imagine a world in which nearly any everyday household item can connect to other devices nearby, generating a wide range of new services and experiences.

Over the years, we have seen a number of visions. For example, a fridge might be connected to the self-service portal of a major supermarket chain to order consumables such as milk and eggs. Cars might communicate their fuel level to their owners’ mobile devices; or technical faults requiring maintenance to a nearby garage. Now imagine this level of connectedness reaches beyond everyday consumer devices; for example, as part of the next generation infrastructure that power cities and public spaces. At any time traffic lights, drains, traffic junctions, and various sensors could broadcast their presence and present status. Digital public interfaces or our own personal devices might offer access to services around us. Soon such scenarios may become reality. It is happening right now, in cities across the world. This is part of the vision of the Internet of Things (IoT) for cities, and we need to develop an understanding of what this vision could be, how it might influence people’s lives, and be shaped by citizens’ real aspirations and needs.

Early IoT and smart city initiatives tended to be preoccupied with functional efficiency and optimisation, and have been steered from the top-down. A key message of CityVerve reflected in this report is that citizens should be stakeholders in technological innovation, especially for technologies deployed in public spaces, or that may otherwise touch people’s lives. Over the past decade, a ‘civic’ movement in Smart Cities has emerged, calling for citizens to become the drivers of smart and livable cities3. This is in contrast to the top-down development of smart city programmes that imply a passive view of citizenship. Citizens should not be passive recipients of standards and technological capabilities that could influence their lives today or in the future. Instead, they need to be in the room when standards and future technologies are designed, and able to consider and assess their future social impacts.

In this report, we review the implementation of human-centred design in the CityVerve demonstrator, a large-scale collaborative effort to deliver cutting-edge IoT services for the city of Manchester together with citizens4. Working with civic groups and use case partners, FutureEverything led on the delivery of human-centred design interventions including targeted human-centred design workshops, broad citizen engagement forums, and experiences and


7

artworks. CityVerve brought together small to medium enterprises (SMEs), large corporations, local government, and the civic sector in one project, testing the opportunities and boundaries of collaborative working. This report critically reviews the process implemented to enable residents, communities and service users to get involved in the design of IoT services that are relevant, useful and address their real needs.

2.1 About the CityVerve demonstrator

CityVerve was a two-year £10m national demonstrator project funded by InnovateUK to develop and test emerging IoT services in four key areas: health and social care; travel and transportation; energy and environment; and culture and the public realm. The CityVerve consortium delivering the demonstrator involved the city council, large global IT companies, including Siemens and Cisco, civic organisations in Manchester, and small and medium-sized enterprises (SMEs) from a variety of sectors.

As a demonstrator, CityVerve aimed to provide an integrative test bed for future IoT applications along the area known as The Corridor, around Oxford Road in Manchester. Contrary to earlier city demonstrators that emphasise specific technologies, CityVerve was focused on challenge areas, each with a set of technology pilots, known as ‘use cases’, within them5. An important innovation outcome has been the CityVerve ‘platform of platforms’, developed as a basis for future technologies to exchange data transparently and enable development of applications using that data. The introduction of a human-centred and citizen-led design process, and of artists and public experiences, can be similarly said to introduce social and cultural openness and interoperability, as a basis for increased citizen involvement in the pilot technologies.

(5) Later in the report we will highlight three use cases in greater detail: BeeActive, Buzzin’ app, and SeeSense.

20 consortium partners including:

3 large city institutions3 multinational technology companies2 universities1 major property management company

15 Community Champions

14 community forums

2 separate targeted design activities

4 themes:

Energy and Environment Travel and TransportationHealth and Social CareCulture and the Public

12,000 individuals reached (including art commission audiences)

240 participants across all workshops

2.2 Scope of this report and constraints to the evaluation

In this report, we present learnings and recommendations drawn from document review, the FutureEverything team, and interviews with use case partners to develop insights for implementation of the human-centred design in future IoT and smart city projects. We chose an

Fig. 1. Engagement in CityVerve

8

evaluation method that relied on detailed interviews with Community Champions, designers, and use case leads to gather ideas useful for future projects of this kind. The evaluation draws on three impact studies in more detail, with the focus on mapping the overall approaches followed, to document lessons for the future.

The report charts the approach to, and outcomes of, embedding citizens in specific use cases within CityVerve and aims to understand the suitability of the human-centred design approach used for enabling the public to influence those use cases. We reflect on the opportunities, but also the challenges, of delivering human-centred design in the project.

This report focuses on some but not all of the human-centred design components in CityVerve. It specifically focuses on community forums and targeted design interventions that contributed towards the development of use cases. Other significant contributions to CityVerve specifically Community Key Performance Indicators (Community KPIs)6 and public art experiences developed using the Open Prototyping framework7, are referenced in the discussion, but are not elaborated on or further developed in this report.

We hope that this report will be useful to anybody who may be involved in similar large-scale multi-partner demonstrator projects. We developed insights for best practice based on the distinctive, and at times challenging, design interventions in CityVerve to provide signposts for the implementation of human-centred and citizen-led design processes in large-scale IoT demonstrators.

(6) Hemment, Drew, Woods, Mel, Appadoo, Vimla, & Bui, Lily. (2016). Performance Indicators (Community KPIs) for the IoT and Smart Cities — A Collaborative Framework for Project Assessment. FutureEverything.

(7) Hemment, D., Bletcher, J. & Coulson, S. (2017) Art, creativity and civic participation in IoT and Smart City innovation through ‘Open Prototyping’. Creativity World Forum 2017. Aarhus, Denmark. November 1-2.

9

(8) See: Hemment, Drew. (2016). How to Design a Useful Internet of Things – Introducing Human Centred Design to CityVerve. Retrieved August 2018, from http://futureeverything.org/news/human-centred-design-cityverve/

(9) See: Hemment, Drew. (2015). Open Prototyping - FutureEverything. Retrieved from http://futureeverything.org/news/open-prototyping-alpha/; Hemment, Drew, Bletcher, Joanna, & Coulson, Saskia. (2017). Art, creativity and civic participation in IoT and Smart City innovation through “Open Prototyping.” Presented at the Creativity World Forum 2017, Aarhus, Denmark.

In CityVerve, human-centred design principles and methods were introduced to help design services and technologies around the challenges citizens face, and that will be usable, useful and likely to be used. This was described in a blog post on the FutureEverything website in 2016, ‘How to design a useful Internet of Things’8.

3. Human-centred design in CityVerve

“ Human Centred Design is an overarching approach that sets the process and frameworks to integrate tools, methodologies and practices to unveil problems, needs, wants, limitations and restrictions of the people who will use the end product or service.

It’s a big picture discipline that focuses attention on improving strategic decision-making by putting the people at the centre of it and continually assessing what’s needed (desirability), what’s possible (feasibility) and what’s sustainable (viability).”

In Santos (2016)

3.1 Human-centred design methods and approach

A distinctive feature of the human-centred design methodology proposed and implemented in CityVerve was that the design process was open to citizens and community participants, and that different technologies were exposed to different audiences, with an emphasis on public events and citizen engagement.

This emphasis on engaging citizens in the human-centred design and in guiding technological innovation was an extension of the Open Prototyping framework9. This has evolved out of FutureEverything’s work in new media arts, and the model of a ‘festival as lab’. Here the focus is to develop and test a concept or process through input of external contributors, and through interventions that are open and accessible to various publics or audiences. The emphasis is on

http://futureeverything.org/news/human-centred-design-cityverve/

http://futureeverything.org/news/open-prototyping-alpha/


10

prototypes that are driven by artists and offer experiences for a large group of people, often in the public realm. The human-centred design described in this report adopted the principles of this framework, which informed the wider approach to introduce creativity, participation and ownership by citizens into the smart city demonstrator.

The design team trialled strategies and methods to enable citizens to access and engage with the IoT demonstrator and technology development. This opening up of the technology development process to citizens, and making it accessible to different audiences, was achieved through a number of methods and activities:

i. Community KPIs: Goals and metrics developed with communities for each of the core themes of CityVerve.

ii. Community forums: A series of open public events, staged at relevant community venues, for people to engage with project themes and share interests and concerns about technologies being developed.

iii. Targeted design interventions: Human-centred design activities tailored to use case needs and engaging a target user group.

iv. Public pilots and demos: A series of public deployments of technologies developed on CityVerve to generate exploration and feedback.

v. Artworks and experiences: A programme of public art commissions creatively engaging with themes and issues in CityVerve

vi. Community Champions: Individuals specifically trained in creative facilitation methods with the core mission to reach out to relevant community groups, primarily through community forums.

Event formats – such as community forums, described in more detail below – were devised to engage citizens around technology concepts and their possible consequences, to build literacy and familiarity, and to do so in an engaging, and non-threatening, way. These forums provided a ‘glue’, tying together various use cases, and their development from early vision, to eventual demonstration. Art commissions, although not the focus of this report, similarly created open and accessible forms of participation that built literacy and encouraged playful exploration around IoT concepts and applications. Together, these methods provided open spaces for dialogue and ‘contact zones’ for the CityVerve project with communities in Manchester around future IoT technologies.

Another way in which this openness was achieved was through the use of creative facilitation. A cohort of individuals skilled in role play, facilitation, theatre, filmmaking - known in the project as Community Champions - were recruited to design the community forums and deliver accessible and relevant citizen engagement. Community Champions were young people who had received training through a highly successful programme at Contact, a youth focused theatre based in the Oxford Road corridor in Manchester. The FutureEverything team devised the Community Champions’ role and framework, building on the cohort’s expertise to enable them to support citizen engagement in innovative ways. Community Champions were provided with briefs for

11

delivering community forums, and given the freedom to respond to them in creative ways.

Community KPIs were proposed and introduced in CityVerve as one way to build meaningful participation by local residents and communities in the governance and design of a major IoT and smart cities project. Community KPIs engage citizens as stakeholders in defining and measuring the success of smart city and IoT projects. In CityVerve, they refer to social metrics for evaluating factors that are crucial to user acceptance of technologies and services. These Community KPIs sit alongside other performance indicators, such as financial return and technology readiness. FutureEverything produced a report in 2016, proposing a practical framework for developing, implementing and assessing Community KPIs for the CityVerve project10. Community KPIs were developed around previously identified “lead use cases”, and as a part of the broader themes. The high-level process, defined and documented in the report included:

• Engaging project stakeholders (e.g. citizens / community members / users / use case leads) in envisioning broader goals and indicators

• Iterating specific goals and indicators for individual use cases

• Assessing finalised goals and indicators with mutually agreed-upon observational mechanisms

• Reflecting on the progression of the goals and indicators

More conventional human-centred design and user research methods, such as personas and user journey maps, were also deployed. These were used in hands-on design interventions aligned to use case needs, to provide an opportunity for participating citizens to influence the development of a particular use case. These design activities were delivered in consultation with use case leads around specific technologies and challenges. FutureEverything worked with use case partners to find and invite citizens with relevant needs, interests, or characteristics to these sessions.

Each use case in CityVerve had a diverse set of stakeholders. The human-centred design process used in CityVerve extended the range of stakeholders involved in individual use cases and in the demonstrator overall. Key participants engaged in the design processes included: project leaders; design experts; Community Champions; citizens and community members; researchers; artists; and representatives of voluntary organisations and service user groups (Fig. 2). The involvement of artists in commissions aligned to the technologies developed and the involvement of Community Champions who helped to facilitate conversations with communities in creative ways were distinctive aspects of the design process on CityVerve.

(10) Hemment, Drew, Woods, Mel, Appadoo, Vimla, & Bui, Lily. (2016). Performance Indicators (Community KPIs) for the IoT and Smart Cities – A Collaborative Framework for Project Assessment. FutureEverything.

12

3.2 Phasing of human-centred design in CityVerve

CityVerve began with an intensive programme of human-centred design activities, that first entailed human-centred design training for use case teams, and engagement with citizens to generate community defined goals. Community forums were introduced, with the aim in the first project phase of generating literacy and shared understanding around critical issues in the IoT, and in the second phase of engaging communities around specific topics and requirements in the use cases. Targeted design activities around user experience and other dimensions of the pilot technologies then led to public trials.

In broad terms, the design process in CityVerve entailed two connected phases of human-centred design and citizen engagement (Table 1). The human-centred design interventions did not follow the Open Prototyping six stage process, which was used in the development of artworks and experiences in CityVerve. Instead they were broadly aligned to the double diamond design process developed by the Design Council (Fig. 3).

Fig. 2. Key participants in the design processes in CityVerve

Project Leaders

Citizens, Community Members, Users

Artists

Representatives of voluntary

organisations and service user groups

Community Champions

Design Experts

ResearchersCityVerve

Stakeholders

13

Scope Discover Define DeliverDevelop Iterate

Phase 1 (month 1 - 8) Phase 2 (month 8 - 36)

Exploring issues around smart cities and the IoT - data security and privacy; accessibility

Using future scenarios to expand on use cases, capturing aspirations and goals defined by the community.

Pilot implementation of Community KPIs - a set of community goals and indicators, that formed a portion of the measures of success for CityVerve overall.

Targeted design interventions delivered around specific technologies and services.

FutureEverything designers were embedded in certain use case teams, after it became clearer which use cases would proceed.

A range of design methods, some experimental, others more conventional, such as personas and user journey maps.

Table 1: Design phases in CityVerve

Fig. 3. Broad phasing of human-centred design interventions in CityVerve overlaid onto the Double Diamond design process model, differentiating between problem exploration and delivery phases.

Community forums PilotsCommunity KPIs

Challenges Insight Definition DeliveryIdeas Embed

HCD training Targeted design activities

The application process for the InnovateUK grant entailed specifying, at the proposal stage, service concepts (use cases) that could be demonstrated along the Oxford Road Corridor. Thus, an emphasis in the early stages of the human-centred design implementation was to challenge assumptions about what kind of solutions are needed and desired, based on user research and citizens’ views. FutureEverything’s aim in the early stages was to support the CityVerve project teams to develop early-stage service concepts informed by citizens’ needs and interests. As the project progressed, the emphasis continued to be on enabling consideration of citizens’ aspirations and concerns around IoT tech, as opposed to an approach solely driven by technical viability and feasibility, alongside concerns for return on investment.

The delivery of the design programme needed to adapt to emerging constraints and requirements.

14

At the outset of the project, the intent was to introduce human-centred principles and methods to CityVerve project teams through training and support. It became clear that more direct and ongoing design intervention and support was required, and so human-centred design activities were reshaped around deeper engagement in a limited number of use cases. As a consequence, fewer targeted design activities took place than expected, and they took place at a later stage in the project.

This report focuses on the approach to human-centred design and citizen engagement that emerged after this point, implemented from month eight of the project. This phase involved regular community forums and targeted design interventions, as two key components.

3.3 A closer look: community forums and targeted design interventions

In the following section, a selection of design and citizen engagement activities employed in CityVerve are described in greater detail. The focus for this report is on community forums and targeted design interventions, and in particular those with which use case leads were most engaged. This is followed by analysis of feedback from the practitioners involved in design, delivery and participation, in these and wider design activities.

3.3.1 Community forums — building literacy and defining community goals

A key element of the human-centred design programme in CityVerve were regular community forums - workshops designed to engage communities in IoT and smart city themes, delivered by the FutureEverything design team and Community Champions. To enable entry points into use cases which span sectors (local government, corporate, design, community) and technology readiness levels, FutureEverything developed a design and engagement method around a series of regular, recurring events connected with the four themes of CityVerve. Events were scheduled on a regular, roughly monthly, basis and served to open dialogues without specific user research objectives. As regular public events, community forums served as a platform for the general public to get involved with the project. Forums always took place in public venues (e.g. a public art gallery) or community venues meaning that people could join activities relatively easily. They took the form of open and varied workshop formats, which were not formulaic, often involving creative, hands-on activities along with talks or demonstrations.

Initial community forums enabled a dialogue with Manchester residents and communities in which a set of project-level goals and indicators were defined by the participants. This was a pilot implementation of Community KPIs, which led to the inclusion of Community KPIs in the overall measurements and indicators of CityVerve11. Community KPIs provided an opportunity for citizens to be involved in the definition of what counts as success in the technology demonstrator. For example, a Community KPI for the Health and Social care theme was to decrease social isolation along the Oxford Road Corridor measured by the uptake of activities in the area.

Other early forums explored broad themes such as data and privacy, lifestyle, necessity and accessibility of smart city technology. Forums explored each theme through discussion of different fictional smart city futures. Descriptive elements of CityVerve technology, embedded in each design fiction, enabled debate and analysis of the technology in terms of its potential impact for citizens.

(11) See: Hemment, Drew, Woods, Mel, Appadoo, Vimla, & Bui, Lily. (2016). Performance Indicators (Community KPIs) for the IoT and Smart Cities — A Collaborative Framework for Project Assessment. FutureEverything.w

15

They were designed to speak to relatively complex challenges, such as social isolation, which CityVerve hoped to address.

As CityVerve evolved, community forums became tailored to use case challenges and themes. Later forums were tailored to specific design stages and outcomes, while others explored high-level questions such as “What does it mean to live in smart city? What do we know about data and privacy? Is the new technology necessary? What does accessibility mean?”. The FutureEverything team captured and reported insights to use case teams for consideration in the development of their technology solutions.

Each community forum was bespoke and specifically designed in order to speak to relevant target audiences in a way that was appropriate to them and the topic. With the range of topics addressed in community forums it would not be possible to have a one size fits all structure. FutureEverything developed a pro-forma to help Community Champions plan forums which helped articulate how event formats and activities aligned to the CityVerve project as a whole (see Appendix: CommunitywChampion Workshop Planning Template).

By focusing on the ‘audience rather than the tech’, community forums provided an opportunity for the ‘framing of future technologies’ appropriate to the respective audience. Forums addressed many of the burning issues surrounding IoT technologies for the community participants. A facilitator noted “when people are talking about data being shared there is ... a lot of fear around that.” As a consequence, many forums addressed high level social dimensions of IoT technologies rather than the usability of an interface, for example. They often adopted surprising and imaginative

16

ways to build relevance, such as a yoga class to discuss attitudes to physical mobility. A mix of talks, interaction, and experiences meant that forums were engaging and fun.

Attributes of CityVerve community forums:

• They build literacy and engagement in project themes and use cases.

• They involve experiences uniquely tailored to the theme or challenge.

• They take place at community venues (or venues relevant to specific target audience)

• They are designed and run by facilitators trained in mixed design and art methods

• They are ‘un-workshops’; the focus was to make them more of a community event or gathering, and less like market research

“The real stuff comes from people activity wanting to contribute to the project. I think that is one of the key areas of the community forums. We are not going out chasing people to make an impact on how their city, or the technology looks. It is them coming to us. It is that buy in from the citizens that is really key; that is where the community forums come in, because we are able to get that buy in from them.”

Use case lead – Sparta Digital

“CityVerve has very clearly set out a plan of these community forums; and done so constantly. So it is not just a case of one every year; or on the cuff type of things; they were almost one every two months. There was a schedule for people to be involved; and they were clearly publicised and out in the public realm.”


3.3.2 Targeted human-centred design interventions — delivering solutions based on citizens’ needs and concerns

Design interventions targeted on particular IoT technologies and services were delivered alongside community forums. The purpose was to help scope and specify technologies through detailed user research and design work. This helped to specify requirements, enhance understanding of users and capture feedback to inform the development of the technologies and services in CityVerve.

Outcomes (design research with bike users)

Identify data on BT hub and CityVerve API of value to cyclists

Understand factors influencing user experience for cyclists

Correlations between those factors and the data from bike light sensors

17

Design interventions were developed for specific use cases, through close consultation and collaboration with use case leads, and were devised to engage relevant end users and members of the public. Targeted design activities for specific use cases included bespoke workshops and design sprints, led by service design experts from FutureEverything. Methods used were, for example, user personas (with the Buzzin’ app) and experience journey maps (SeeSense) to address specific challenges and applications arising in the IoT technologies and services. For example, targeted design activities helped to establish use cases for data gathered from sensor-equipped bike lights (see SeeSense – Improving cyclists’ experience in the city) and clarified user journeys for an augmented reality (AR) app (see Buzzin’ app – Discovering the city).

User testing and hands-on user research explored user acceptance and validated technology design ideas. By engaging relevant community groups and members of the target audience, design facilitators helped the use case leads understand and empathise with users’ concerns. In the process, these activities helped to specify target users for specific use cases and the value the application may unlock for them.

The aim was to generate data and insights to enable development of the technology in CityVerve. There was also an aim to generate wider visibility for CityVerve, and to inspire wider public debate about the implementation of IoT and smart city technology in Manchester.

Attributes of targeted human-centred design interventions:

• Input of use case team to determine product-related user research

• Involvement of use case team to set base-line of challenges and develop a plan of action

• Invitation and selection of citizens that match relevant audience / user profile

• Use of engaging activities that immerse participants in the issue

• Focused report back to use case lead

“FutureEverything designed and delivered a full-day workshop to Sparta Digital, which gathered team insights on potential audiences for City Concierge. Sparta Digital’s team members were involved in different stages of the workshop and each member offered valuable perspectives based on their roles in the diverse stages of production. This gave us an opportunity to delve into the audience, project requirements and timelines collaboratively.”


18

(12) Report on Community KPIs and on SmartCitizen (Hemment et al., 2016)

(13) See Appendix: Interview guide

(14) See Appendix: Comments from event attendees

4. Evaluation of human-centred design in CityVerve

4.1 Evaluation methods

4.1.1 Materials considered

For this report, we reviewed a large set of documents, including two previous FutureEverything reports12, 18 formative and outcome reports from targeted user research and community forum events, CityVerve presentations, and 28 blog posts by FutureEverything about CityVerve, dated between 2015 and 2018. These documents helped to construct a timeline of events and activities across the project lifetime.

The document review and input by the FutureEverything design team was used to develop a series of interview questions for semi-structured interviews. Eight interviews were held, looking into specific use cases in greater detail with interview participants comprising designers, Community Champions, and use case leads. Four interviews were with use case leads and four with design facilitators across three use cases - the BeeActive app, the SeeSense e-bike project, and the Buzzin’ app. Interviews followed an interview guide13 and offered room for free conversation while covering specific questions around challenges encountered and impacts arising from the delivery of specific activities.

Citizens’ voices were represented through the contents of the formative and outcome reports, and feedback was gathered from engagement participants by FutureEverything during the delivery of human-centred design activities14. Due to time limitations, the evaluation of this data capture was limited.

4.1.2 Analysis methods

Transcripts were prepared for each of the eight participant interviews. These transcripts, along with all secondary data material, were imported into a qualitative analysis tool to support coding and synthesis of the material.

Qualitative materials were analysed in two primary ways. Firstly, through the systematic mark-up of events, time points, and participants, we formulated timelines of events for the CityVerve project and the three individual use cases. Each use case was analysed in greater detail by constructing

19

a sorting system that described each project (use case)15. Secondly, interview questions were analysed to derive a set of key themes. These themes included the value human-centred design added to use cases, what participants were proud of, any lasting impacts, what the challenges were, what participants wanted more of, and finally, what insights participants gained on human-centred design. Along those themes we coded quotes that presented lessons learned. In a second step, those coded quotes were organised to describe common points raised.

4.1.3 Data reporting

Intermediary outcomes from data analysis was summarised in a brief for conducting interviews. The brief contained subsections for key questions in the interview guide. Under each section, a summary table listed the themes on the left hand side, as well as the frequency by which these themes were raised by design facilitators and use case leads respectively. For the findings in this report, responses to the question of lessons learned were of particular relevance. This underlies content in a following section of this report Insights gained - lessons learned. Case studies were written up separately (next section) helping to relate insights from different design activities to specific use cases.

(15) In this case, those were: aims, partners, product description, project description, tech involved

20

5. Outcomes of human-centred design interventions

The following section presents three case studies of human-centred design interventions in CityVerve use cases and their outcomes, evaluated using methods described above. The case studies cover different project themes and types of design activity. Each case study reflects on the aim of the use case, partners and participants involved, and the outcomes of the design activities.

5.1 BeeActive – Encouraging an active lifestyle

BeeActive emerged in the Health and Social Care theme and involved a collaboration between academics from the University of Manchester and developers at Clicks + Links, a Manchester-based start-up focused on virtual reality experiences. BeeActive uses a smartphone app to encourage people to be more physically active. It works by nudging or prompting people to accept missions, created by the community, based on their location, time of day, or the users’ behaviours - prompting them to become more active at a particular time or at a particular opportunity, for example during their journey to work.

FutureEverything facilitated open community forum events as well as a targeted persona workshop to help determine likely user groups for the app. Participatory activities in the open-ended

Activities performed

Two community forums, one involving a creative sports activity to discuss active life in the city

Human-centred design workshops including review of user groups through development of personas

21

community forums helped to determine what type of triggers or prompts to use to get people more engaged without referring to the app - focusing on the outcome rather than the technology. For instance, the community forum ‘workout buddies’ engaged participants with discussions around health, warm-up and physical activity, physical skill sharing.

By discussing these experiences, the team gathered insights into health and fitness; talking about why people exercise, what motivates them, what they currently do, what their barriers and facilitators are, and things people could do to become more active. Participants in this community forum greatly enjoyed themselves and delivered insights valuable to the use case partner.

“I think it was really useful to have quite open and broad ideas from people about what they thought they might like to see in a physical activity app, for example. It was quite useful, because I think the sessions were, as I say, fairly open. They weren’t too prescribed or too focused which was really useful. I think it was probably more engaging and probably more encouraging for people to talk more openly about a broad topic rather than being too focused.”

Use case lead – University of Manchester

(16) Central Manchester University Hospitals, University of Manchester, Manchester Metropolitan University, Manchester City Council, Manchester Science Partnerships, Royal Northern College of Music

(17) University of Manchester, Manchester Metropolitan University, Royal Northern College of Music

(18) All primary school children in Manchester are weighed and measured each year and CHAMP is a programme run by CMFT which asks parents to sign up and encourage healthy behaviours in their children

In terms of targeted design activities - BeeActive was conceived as being for four broad cohorts - staff who work for the large employers along the Oxford Road corridor16, students from local universities17, CHAMP parents18, and school-age children. Through a persona development workshop facilitated by FutureEverything, the use case team developed greater clarity about the typical person from within each of those groups - what might motivate them, what might engage them, and what might put them off. Successful implementation of human-centred design

22

interventions enabled smoother development of the technological solution.

5.2 See.Sense – Improving the city experience for cyclists


Human-centred design workshops with cyclists and use case partners focused on detailed user journey mapping to identify common concerns while travelling the city and how those might be reflected in the sensor data

In the transport theme, the e-bike use case involving BT and See.Sense sought to establish IoT services that could benefit mobility and deliver safer streets. The use case was based around an intelligent bicycle light with proximity and motion sensors, and connected with the CityVerve data infrastructure to explore how crowdsourced data from cyclists using the lights could help inform investments in cycling infrastructure, which in turn could encourage more people to cycle. The challenge for the use case team was to establish the value to individual cyclists of data produced by those lights, which ultimately could drive wider application of such technology.

The bike light senses environmental conditions, including vibrations induced from road surfaces and uses its proximity sensor to flash brighter and faster in riskier situations such as road junctions and roundabouts. The sensors can also anonymously gather user inputted data such reports on the quality of the road surface as well as cycling routes, accidents and near-miss events – providing qualitative data that is highly accurate and allowing the city to be mapped as never before. Anonymised data is aggregated on BT’s IoT Information Exchange platform, making it available to the wider CityVerve ecosystem of innovators.

To help understand how to build data visualisations which might be useful for cyclists who use the intelligent bike light, FutureEverything arranged targeted design workshops involving the development of user journeys and exploration of the road experience of cyclists. In a workshop, a GoPro video of a common cycle journey in Manchester, from the Oxford Road corridor to the Northern Quarter, was shown. Participants collaboratively prepared a user journey map reflecting on positive and negative aspects of the ride. Participants were asked to pay specific attention to incidents when the cyclist had to divert or respond to an external influence, such as a car or an obstruction on the road. Notes were pulled together into a large consensus user journey map, through which participants explored the causes of each incident.

The consensus user journey map was very valuable to the use case lead and cyclists involved as it clearly communicated insights on specific causes for high and low points on a cyclist’s journey. In the user journey map, each peak and trough served as a point for further conversation, helping build a robust profile of what made the short journey good or bad from the point of view of cyclists. This visual approach helped to reflect on the data reported by the bike lights and to establish events that can be traced through the data. For the use case lead, during an analysis workshop on

23

the user research, it resulted in an understanding that they have a strong technological offering for their users and a greater sense of what could be involved in “intelligent route planning”.

“When I see things in this kind of context, I’m fairly visual, so I can think of what other linkages I can pull in. What other data sets might be helpful. It just helps to gel together much, much better. So for me that’s workshop was extremely valuable.”

Use case lead – BT

5.3 Buzzin’ app – Discovering the city around us


Two human-centred design workshops with persona activities; and one with detailed user journey mapping to establish use cases

A community forum event combined with a walking tour of a public event to test the prototype in real life.

Public pilot as part of Manchester Pride

Another use case in the transport theme set out to respond to a question about future ways citizens would interact and engage with the city around them. The Buzzin’ application was developed by Sparta Digital, a Manchester-based SME, along with city organisations, such as Transport for Greater Manchester. Built around an augmented reality experience, Buzzin’ integrates with the real-time CityVerve API - a project-wide data application which included granular information on street-level furniture, transport, and visitor amenities, to be more than a wayfinding application.

Through targeted design activities, FutureEverything identified opportunities to create both immediate and long-term impact; helping Sparta Digital refine the value proposition of the Buzzin’ app. FutureEverything delivered persona development workshops that helped Sparta Digital to understand key user personas within the wide range of possible audiences, including commuters, shoppers, and tourists and their motivations to use the application.

To explore the use of the Augmented Reality prototype in a real-life situation, FutureEverything facilitated a walking tour using the Buzzin’ app at the Community Forum in December 2017. Community Champions took 25 members of the public around the Manchester Christmas Market to road-test the Augmented Technology for discovery and wayfinding. Different challenges, including a treasure hunt and engaging with stall holders, were set. After the event, participants were invited to respond to evaluation questions using Mentimeter, a real-time voting app. Event participants were excited about the potential of digital wayfinding and the event helped Sparta Digital validate their prototype application.

24

Through engaging in these design activities, Sparta realised that they had built two different products, an event-based application and a city discovery application. The team realised that the products have very different user journeys. As an outcome, Sparta chose to develop a distinct value proposition aimed at locals who want to learn about their city. These human centered design activities with users changed Sparta’s own design approach, which did traditionally not involve structured human-centred design.

“The CityVerve project allowed us to really experiment and see what people want; and what we are capable of doing as well.”

“I think that what we have learned as a company is that we need to make sure that we are carrying out this framework to ensure that the solutions we are developing are overcoming challenges and problems people face. This is what the human-centred design approach is about.”


25

Evaluation was based on interviews with eight CityVerve participants – use case leads, designers and Community Champions – and a review of project documentation. Here, we summarise some of the lessons learned for delivering human-centred design in an IoT demonstrator that could be of help to future technology innovation projects.

Some of these lessons speak to overall project structure and design, some to specific activities, methods and tools. The evaluation highlighted some of the benefits of implementation of human-centred design in CityVerve, and revealed a gradual shift in the design approach taken across different use cases during project delivery. It also highlights some of the challenges encountered, such as those stemming from working across a broad set of partners with different cultures and sizes, and also the changes in the work plan documented in this report.

The results of the evaluation are presented as signposts for the implementation of human-centred and citizen-led design processes in large-scale IoT demonstrators. We developed the following insights and recommendations from the interviews, which asked participants about challenges encountered and lessons learned when developing the use cases in CityVerve with communities involved. There was a diversity of observations, with many interviewees offering recommendations on how to address the challenges they faced. The lessons learned represent a synthesis of common topics that surfaced in several interviews. The results include a number of actionable takeaways and recommendations that we developed on this basis. While each large-scale demonstrator is unique, these insights could be widely applicable to other similar multi-partner projects, too.

6. Insights gained – lessons learned

26









Table 2: Lessons learned for future IoT and smart city demonstrators

27

Overall, participants we spoke to looked forward to the future collaborative potential for technology innovation engaging citizens in Manchester.

One recommendation is to involve community groups in defining specific challenges or use cases, during the project, or when the project is designed. It is important to be led by, and engage people through, challenges that are relevant to people, and avoid a wholly technology-driven conversation. This can enable use case teams to understand and specify the needs they address in the pilot technologies.

There is value in identifying specific communities or groups as part of the consortium formation and working with them for the whole project timeline. Where they are constituted in a legal entity, it is possible to make community or interest groups formal members of project consortia.

Community KPIs are a methodology that can be used to define community goals and indicators, during project design, or project delivery. Use case leads and design experts reported that the Community KPIs were useful to inform a high level aims and goals for the project as a whole. Equally, it is important to build in flexibility to those goals in the event that implementation of individual use cases does not materialise, as was partially the case in CityVerve. The role of Community KPIs can be to document aspirations and reflect on those aspirations long after a project completes.


28

At the outset of the project, there was low understanding, and often low interest, in human-centred design, among consortium partners and members of the public alike. The design team had a dual challenge of communicating the human-centred design process, and its possible outputs. It was necessary to be both advocate and expert, champion and guide.

For use case teams, it was necessary to communicate outcomes that could be achieved by tailoring human-centred design activities and engagement with the public to use case needs. For members of the public, it was important to discover relevance, and build literacy and understanding, in both the human-centred design, and the IoT technologies.

It is important to acknowledge that project partners benefit differently from human-centred design. In a multi-partner consortium with a range of company sizes, smaller organisations without dedicated in-house resource could best benefit from engagement with design activities. A facilitator noted “For me, the ones where it had seemed to work, were the slightly smaller companies or tech firms, who have really got on board with [human-centred design].”

Larger partners can take a longer time to adapt internal work processes and approach to design since they already have well-resourced teams with established ways of working. At the beginning of a project, it is advisable to takes time to develop understanding of the differences across partners and how best to incorporate them into a differentiated programme of human-centred design activities. Culture setting and building consensus around principles and methods to be adopted at the beginning is invaluable, and can be consolidated by defining a pledge or statement


for partners to sign up to in order to facilitate mutual understanding.

A use case that successfully communicated benefits, by building on understanding of the citizens’ challenges, was Chronic Condition Management which developed smart inhalers for individuals with Chronic Obstructive Pulmonary Disease (COPD). To community participants, involvement in the project made a difference to their quality of life, which in turn provided a strong motivator and selling point to communicate the project to their communities.

29

Design specialists can help to support project teams who may not have the skills or capabilities to deliver a structured human-centred design process themselves. In CityVerve, a team of designers provided a resource to bring together different activity streams and audiences. We learned it was not just about providing lessons and workshops to explain the principles of human-centred design to individual use case teams, but it was more important to offer tailored human-centred design activities based on the needs of different use case teams. Consistent involvement of a design team is essential to follow up design activities and help use case teams interpret outcomes of dialogue with citizens.

Engaging specific communities in a project like CityVerve requires building working relationships and trust, which come over time. Engagement is not about flying in and doing something and then flying out. It is about partnership, working with people over a period of time. Interview participants noted that it took a considerably longer time to build up trusting relationships with specific communities than first envisaged. The benefit of building a strong community of project advocates through regular community forums emerged as significant in CityVerve. Community Champions noted that consistent follow-up is easier to achieve with ‘mini sprints’ of activities involving the same community and challenge over a short timescale. Identifying specific community groups and engaging them regularly can make human-centred design with citizens easier.

It is important to plan for time and effort required to arrange public activities. Designers noted that “If you’re catering something so unique, I think the lead in time needs to be greater”. To retain the benefit of tailored events that work well to engage and deliver insights while working with limited resources, arranging a series of linked activities involving a similar audience over a longer period of time could be considered. Dedicated facilitators can ensure


that lead-times for suitable engagement and design activities are manageable. It could help reduce fluctuation in in project engagement as a greater regular following provides greater certainty of the audience for events and could reduce time required for introducing the project rationale and benefits of participation.

30

Time for user research and building stakeholder relationships needs to be built in at the start of a project, as does time and resource to implement findings. It is common for some changes of scope to be expected in a project, thus it is advisable to use the early period of a project for refining challenge areas and priorities together with the members of the public and reviewing how use cases could help refined challenge areas. Defining the technology solutions before the project begins can inhibit the ability to accommodate user needs. It can be helpful to this end to adopt agile methods and include stage gates where stalled projects can be removed or objectives reset.

In CityVerve implementation, much early effort went in to use cases and various goals which could not be realised later on. This was exacerbated in CityVerve by designation of lead use cases across project theme areas, many of which needed to change at a later point. The insight is that there is a benefit in being more open to emerging challenges and insights early on.

Interviewees indicated that the evolution of use cases as described in the initial project application needed some time to materialise. Looking back at the overall design process of CityVerve, interviewees recommended specific ways in which design and community engagement activities could have been rearranged or phased. “The workshops were very thought provoking and got people to think about the city and what might change. It would have been useful to have them first so that you had the participants ready for the community forums where they can really go in depth into each subject.”

The citizen engagement in CityVerve began by reaching out to community and group leaders within the project area to understand issues and concerns that overlapped with themes within the project. Critically, it did this through engagement activities in community venues located in the focus area of the project, and facilitated by Community Champions, who are themselves members of those communities.

In CityVerve, each community forum was uniquely designed to fit the challenge and the audience likely to attend. To evoke interest and insightful responses, events consisted of a mix of talks, hands-on activities, and activities out on the streets. Planning unique event formats required a considerable amount of time and effort but was required to avoid workshops that were bland and purely instrumental.

Conversely, it was important to involve use case teams in the design and priority setting for community forum events and targeted human-centred design activities. Events and activities need to speak back to specific use case goals; “so it’s possible that just before a community forum event […] just asking the question ‘Is there anything specific that you’d like us to cover within this session?’ […] can be quite useful.”

To build trust and meaningful engagement of citizens in the design process, interviewees, and in particular the project facilitators, suggested it is essential to “go to them” rather than simply communicate outward. “You cannot just advertise and expect people to come. You must go to the people you want to engage. So for instance, if you want to engage students, you need to engage in an environment that students are comfortable in and not just advertise and expect them to come to you. […] You might then find that people start to engage or move away from that community space or area further down the line for an extended engagement.”



31

In the early phases of CityVerve, the emphasis in project-wide communication was on making the CityVerve brand recognisable. Interviewees made the case that awareness raising and marketing should be secondary to citizens’ needs, and defining challenges around those. Brand and recognition for the project then follows over time. The primary emphasis in building meaningful engagement should be the challenges citizens face.

In CityVerve, building a cohort of Community Champions and establishing a strong network of intermediary organisations around the Oxford Road Corridor helped to build relationships across communities. Some gaps and barriers were encountered, such as the lack, at project start, of data or understanding of the local population or therefore what constitutes a representative sample for user research. To accommodate this, in CityVerve there was a focus on engaging intermediary organisations.

In a large consortium, it is important to build a framework and practices to enable building community links and sharing contacts in useful, and compliant, ways. It takes time to develop contacts and trust with relevant community groups. In addition, the General Data Protection Act (GDPR) creates a new environment for data management and use and therefore this aspect requires careful planning.

The CityVerve project created a subscriber list for regular mail outs about human-centred design activities. At times, a single subscriber list with restricted access can be limiting and it is advisable to define arrangements early on in a multi-partner consortium. Sharing contacts between partners, where there is a clear opt in, can support the development of meaningful relationships and community links. Cooperation across partners can avoid an uncoordinated stream of emails and requests. Agreements on how contacts are shared is also essential in the


preparation of public events, especially where an audience that is representative of a user or community group is the required outcome.

One of the Community Champions noted “Try to engage people as early as you can and try to make the most of the groups that you and the partner organisations already have contact with. Not that you want to utilise the same groups of people over and over again so that they become tired, but at the same time it makes sense to make the most of the contacts you already have.”

32

Like any demonstrator, CityVerve provided an environment to test technologies that are not yet market-ready. Especially at the beginning, demonstrator projects can appear messy until requirements and use cases are established and gradually resolved through prototyping and technology demonstrations. Many of the technologies to be demonstrated may be at various stages of development; and other challenges may come along the way. For example, the e-bike and talkative bus stop use cases waited for core technology to be available before carrying out any human-centred design activities. Towards the end of the official project timeframe, respective prerequisites were in place and targeted design work began later than expected. With time and availability of application prototypes, the overall narrative of the project became clearer to the various external audiences. Often the end of the demonstrator is also the beginning of much follow-on work.

“Once things started clicking into place, and you can see that in the past six months in the project and on the events that happened last month, we are finally showing things, the stories are consistent. It says where they interact from- you can jump from one demo to another, you can talk to a citizen for three hours showing them all we’ve done in a vocabulary that makes sense to a citizen. You can do the same thing to a company, you can do the same thing to a technical person. That shows the richness of the project.”

Community involvement in defining challenges, and the promotion of challenge-based projects that impact the public, can raise expectations for solutions to be delivered. Innovation and demonstration projects usually do not deliver full implementation of services, and people will be disappointed if the project does not deliver what they expect. Transparency and clear communication of project scope and constraints is therefore important, to avoid damaging trust and participation in future projects.

Open and informed debate about the limits and consequences of technologies can help to build literacy and trust. It is important to address anxieties people may have about a project involving technology development. Engaging people in their own space can also help to build confidence in addressing anxieties that may arise in a technology development project. In CityVerve, critical discussion about the IoT and smart city was promoted. Experienced facilitators can keep critical debate constructive, and manage situations where a vocal individual or minority, who may have a grievance with existing public services, dominate the discussion. As noted by a community champion, “In terms of why people might come to a workshop like that, they might have thoughts and ideas about the way the service is run. Although the aim is always positive, people do come with certain grievances sometimes.”

Consistent follow-up and documentation throughout the project helped to capture such concerns to feed them into the design process, to elicit requirements, and understand needs and concerns. All challenges raised by citizens can be fruitful grounds for design and should not be put to one side. Where conflicts do arise, there needs to be clear communication of the purpose of public events and design activities, so participants understand the purpose and the likely outcomes.



33

We have generated these recommendations for implementation of human-centred and citizen-led design processes in large-scale IoT demonstrators. CityVerve has furthermore established new connections and collaborations, and set a baseline for future technology projects in the city of Manchester.

As such, the CityVerve project lay important groundwork for continuing the implementation of IoT services which have a consideration and involvement of citizens. CityVerve illustrated the challenges of working across a broad set of partners with different cultures and sizes. Not all project challenges can necessarily be avoided, such as delays arising from formal public tendering processes, or the unavailability of specific infrastructures, but these challenges can be anticipated and prepared for. This can be done in project design, or alternatively at project inception, by building in time and resource for a preparatory stage (the ‘prestart’). To successfully and meaningfully introduce a human-centred and citizen-led approach, a component of this preparation needs to be building consensus and shared understanding around a robust framework for human-centred design.

This report has highlighted that there are significant challenges inherent in delivering such a large scale project through close engagement with communities and residents, and documents ways in which the project team addressed these challenges. The involvement of Community Champions helped to facilitate connections with specific groups and generate regular conversation with specific citizens. It was important the design of the project plan responded to both the interests and availability of the citizens, and the needs and time of the use cases teams. This made it possible to convene groupings of citizens around a challenge in a specific time, in a way that did not treat them as sources of information to be exploited, but as stakeholders to be consulted. The phasing and tailoring of design activities provided a strong platform to develop human-centric IoT.

In the piloting of such an approach in CityVerve, the time taken to build this shared understanding of, and commitment to, such an approach made it necessary to be flexible and accommodate necessary adjustments in delivery. By the time it was clear which cases would proceed and which not, the delivery of human-centred design needed to be adapted, and shifted from implementation across the project, to tailored design and user research activities in individual use cases.

7. A note for future large-scale demonstrators

34

Future projects would need to anticipate that it takes commitment and time to build meaningful relationships between project team members, and with communities and residents. In CityVerve, it required time and effort to build those relationships and establish links with relevant community groups. This would where possible start before and continue beyond the project period. It is advisable to build processes, contacts and capacity to productively manage the various interactions early on.

We have seen that from project design or pre-start stage, it is useful to set direction around community needs, focus the project on relevant challenges, and build communities of interest around the project. Early and meaningful engagement with relevant communities helps people frame challenges for their city and how they could be addressed. Over the timeframe of the CityVerve project, communities and residents that participated benefited from a space to discuss concerns and relevance around future technologies, and helped the technology leads consider their voices in the development of the IoT services.

35

It is important to recognise limitations that apply to this evaluation of human-centred design in CityVerve.

Due to the complex nature of such a large, multi-stakeholder project such as CityVerve, and various scheduling issues in the delivery of work packages, FutureEverything had to revise the scope and phasing of human-centred design activity in the project. This introduced pressures on delivery, which in turn constrained the extent and depth of the evaluation.

The evaluation methodology focused on a document review and eight stakeholder interviews, with emphasis on three selected case study projects, and some, but not all, of the design methods and activities implemented in the project. Due to constraints in resource and time, we were not able to appraise through quantitative measures the effectiveness of community forums and targeted design activities, in order to detail in depth the impact those activities had on design decisions.

These shifts in overall project development also resulted in some stages of the Community KPI process being truncated or curtailed, meaning that community goals and indicators were not rigorously validated. Also, other activities such as the artwork commissions that engaged a substantially larger audience within the city and beyond could not be considered. Evaluation of both Community KPIs and the artworks is out of scope for this report

8. Limitations to the evaluation

36

The overall aim of the design implemented by FutureEverything in CityVerve was to enable IoT services to better address the real needs of people, and to enable citizens to play a part of setting the direction of future IoT services.

We hope the insights and signposts presented in this report, along with rationale to the approach adopted, will inspire and equip others to seek out the benefits of human-centred design and meaningfully engaging citizens in technology innovation projects.

We began this report by noting the preoccupation of early smart city initiatives with functional efficiency and optimisation. In CityVerve, we have seen the piloting of a different approach, based on the aspiration to give citizens of Manchester the opportunity to shape the future IoT for the city. Across CityVerve as a whole, FutureEverything has demonstrated principles, methods and tools that have since been disseminated more widely. The art and public experiences developed by FutureEverything for the Culture and Public Realm strand have been toured elsewhere. The first CityVerve art commission, Every Thing Every Time by Naho Matsuda has been restaged at the Great Exhibition of the North 2018, with potential commissions in Hong Kong and Japan. The method to involve citizens in defining and measuring criteria for success (Community KPIs) is being built on in other cities, notably Edinburgh.

In a number of ways, the CityVerve project has developed blueprints, infrastructure and collaborative networks for the future. In the technology developed in the project, this includes the key innovation of the ‘platform of platforms’, which can be a basis for future technologies to exchange data transparently, and the development of applications using that data. The introduction of a human-centred and citizen-led design process can likewise underpin a model of social and cultural openness and interoperability in the development of future technologies.

The increasing maturity of many technologies underlying the IoT and the smart city will enable greater engagement with citizens around their implications and uses. It is likely there is a need to carefully balance empowerment and automation, or seamlessness and friction. Citizens need to become more centrally involved in addressing complex concerns, such as data privacy, surveillance and trust. To achieve this, such projects need to reach beyond people already active in technology, to residents and service users who have no particular interest in technology and who, more than likely, have never heard of the ‘Internet of Things’ or ‘smart city’.

When increasing numbers of entities in the city can become interactive – sending and digesting data generated through the interactions of people in the public domain – it is paramount to adopt

9. Final notes and next steps

37

effective approaches to build literacy, relevance and agency for citizens, along the way to designing services that are useful, usable, and likely to be used. We look forward to a future of IoT services developed with the aspirations, agency, and desires of citizens at the centre.

38

10. List of referencesAshton, Kevin. (2009). That “internet of things” thing. RFID Journal.

Balestrini, Mara. (2017). A City in Common. UCL, London. Retrieved from https://bit.ly/2nWsM7j

Bonney, Rick, Cooper, Caren B., Dickinson, Janis, Kelling, Steve, Phillips, Tina, Rosenberg, Kenneth V., & Shirk, Jennifer. (2009). Citizen Science: A Developing Tool for Expanding Science Knowledge and Scientific Literacy. BioScience, vol. 59, no. 11, 977–984.

Caird, Sally, Hudson, Lorraine, & Kortuem, Gerd. (2016). A Tale of Evaluation and Reporting in UK Smart Cities. The Open University.

European Commission. Collective Awareness Platforms for Sustainability and Social Innovation (CAPS). Retrieved from https://bit.ly/2NKs1tk

FutureEverything. History. Retrieved August 2018, from http://futureeverything.org/history/

Future Cities Catapult. (2018). Smart City Demonstrators. Future Cities Catapult. Retrieved from https://bit.ly/2L1oEQy

FutureEverything. (n.d.). Smart Citizen Manchester - FutureEverything. Retrieved from http://futureeverything.org/projects/smart-citizen/

Gilbreth, Frank B. (1911). Motion study. New York: Van Nostrand.

Hemment, Drew. (2004). Mobile Connections. Futuresonic. Urbis Museum.

Hemment, Drew. (2006). Locative arts. Leonardo, vol. 39, no. 4, 348–356.

Hemment, Drew, & Townsend, Anthony (Eds.). (2013). Smart citizens. FutureEverything.

Hemment, Drew, Smith, Kevin, Tait, Julian, & Dornan, Anna. (2013). DataGM: Curating an Environment for Change. In H. Melkas & J. Buur, (pp. 194–198). Presented at the Participatory Innovation Conference, Lahti.

Hemment, Drew. (2015). Open Prototyping. FutureEverything, Manchester. Retrieved from http://futureeverything.org/news/open-prototyping-alpha/

Hemment, Drew. (2016). How to design a useful Internet of Things – Introducing human centred design to CityVerve. FutureEverything, Manchester. Retrieved from http://futureeverything.org/news/human-centred-design-cityverve

Hemment, Drew, Woods, Mel, Appadoo, Vimla, & Bui, Lily. (2016). Performance Indicators (Community KPIs) for the IoT and Smart Cities — A Collaborative Framework for Project Assessment. FutureEverything.

https://bit.ly/2nWsM7j

https://bit.ly/2NKs1tk

http://futureeverything.org/history/

https://bit.ly/2L1oEQy

http://futureeverything.org/projects/smart-citizen/

http://futureeverything.org/projects/smart-citizen/





39

Hemment, Drew. (2017). Art in the IoT and Smart City demonstrator, CityVerve. FutureEverything, Manchester. Retrieved from http://futureeverything.org/news/art-iot-smart-city-demonstrator-cityverve/

Hemment, Drew, Bletcher, Joanna, & Coulson, Saskia. (2017). Art, creativity and civic participation in IoT and Smart City innovation through “Open Prototyping.” Presented at the Creativity World Forum 2017, Aarhus, Denmark.

Hill, Dan. (2013). Smart Citizens Make Smart Cities. In D. Hemment & A. Townsend, Smart citizens. FutureEverything.

ICO. (n.d.). Guide to the General Data Protection Regulation (GDPR). Retrieved from https://bit.ly/2A10ayF

O’Brien, Jeffrey. (2009). IBM’s Smarter Planet: the roadshow. Retrieved from https://for.tn/2NJih2K

Salinas, Lara, Coulton, Paul, & Dunn, Nick. (2016). Using Game Design as a Frame for Evaluating Experiences in Hybrid Digital/Physical Spaces. Architecture and Culture, vol. 4, no. 1, 115–135.

Santos, Daniel. (2016a, December). Four Lessons From Last Four Months. Retrieved from https://bit.ly/2L7XmrP

Santos, Daniel. (2016b, December). Incorporating Human Centred Design in CityVerve. Retrieved from https://bit.ly/2NGzE48

Scott, Michelle, Woods, Mel, Hemment, Drew, Coulson, Saskia, Polvora, Alexandre, & Nascimento, Susana. (2017). Report on the making sense framework and assessment of participatory strategies. Retrieved from http://making-sense.eu/publication_categories/toolkit/

Thomsen, David. (2012). Why human centered design matters. Wired. Retrieved from https://bit.ly/2f9dOF8

Townsend, Anthony. (2013). Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia. W. W. Norton & Company.

van der Vleuten, Ruben, et al. (2015). The IoT Design Manifesto. Retrieved from https://www.iotmanifesto.com/

Waag Society. History. Retrieved from https://waag.org/en/history

Weiser, M. (1991). The computer for the 21st century. Scientific American, vol. 265, no. 3, 94–104.

http://futureeverything.org/news/art-iot-smart-city-demonstrator-cityverve/

http://futureeverything.org/news/art-iot-smart-city-demonstrator-cityverve/

https://bit.ly/2A10ayF

https://bit.ly/2A10ayF

https://for.tn/2NJih2K

https://bit.ly/2L7XmrP

https://bit.ly/2L7XmrP

https://bit.ly/2NGzE48

http://making-sense.eu/publication_categories/toolkit/

https://bit.ly/2f9dOF8

https://bit.ly/2f9dOF8

https://www.iotmanifesto.com/

https://www.iotmanifesto.com/

https://waag.org/en/history

40

The following interview questions were drawn on as prompts in interviews with eight key stakeholders.

Appendix: Interview guide

Stakeholder Questions

All participants • Could you briefly describe your involvement on CityVerve? • What achievement are you most proud of?

Designers / Community Champions

• Looking back at the early phase of the project, what were the challenges for incorporating human-centred design in CityVerve? • Any key lessons to share? • How do you best select Community Champions and what do they need to be successful? • Could you share an example, how community forums and engagement of the public influenced the use case? • Could you share an example, how targeted design activities influenced the use case?

Use case partners • What made you get involved in CityVerve? • Could you share an example, how engagement of the public through community forums influenced your project? • Could you share an example, how targeted design activities influenced your project? • What aspects set citizen involvement in CityVerve apart from others demonstrator projects you have been involved with?

All participants • What have you learned about citizen involvement in designing IoT products? • After engaging with CityVerve, what is ‘smart city’ to you? • How has CityVerve enabled you to shape the ‘Internet of Things’ for Manchester? • What three things could there have been more of in CityVerve?

41

This document was used to provide a framework for organisers of community forums to help support planning and development of community forums. It helped to ensure community forums followed CityVerve’s project criteria.

Appendix: Community Champion Workshop Planning Template

Logistics

Workshop Date

Workshop time (including prep time on the day)

LocationIs your chosen location in the CityVerve project area? Why have you chosen this location? Is it relevant to your target audience?

Existing relationship with venue? If yes please provide detailsIs the venue somewhere that you have previously used? Do you already have contacts there? Is this a common venue for your target audience?

Target Audience Who do you want to engage with at the workshop?

Audience relevance to CityVerveHow is your target audience relevant to CityVerve? What benefit will this engagement bring to the project?

Number of participants What’s the minimum and maximum number of attendees you want to have at your workshop?

How will you get people to attend/market the workshop?Do you need to get people to sign up? How will you do this? Do you need an eventbrite? How will you promote the workshop and ensure you get the right people at the event? What’s your selling point?

42

Use Cases/Business Scenarios

What CityVerve Use Cases/Business Scenarios will you be incorporating?List the CityVerve Use Cases and Business Scenarios that you will be featuring at the workshop. Do you want to showcase any technology?

What Smart City/IoT themes will you be addressing?Are you addressing any overarching themes? Will you be asking any specific questions?

How does your workshop fit into CityVerve?Does your workshop fit into a CityVerve theme? Does it fit into the project in a different way?

Evidencing

How will you capture audience insights?Are you going to ask participants to fill out worksheets? How will you capture discussions and conversations? What prep and materials do you need for this?

Do you feel well equipped to present and discuss CityVerve and its technologies? How can FutureEverything further support you?

What digital content will you be using/creating?Will you be posting any digital content on social media before/during/after your workshop? How will you share this with other Creative Experts?

Community KPIs

What Community KPI(s) will you be assessing?Are you assessing the Community KPIs. If so how?

How will you assess the Community KPI?How will you capture insights? Will you quantify opinions? How will you address the KPI?

43

Workshop

Workshop formatWhat is your workshop format? Is it a workshop or something different (role play, future scenarios, open discussions etc.)?

Workshop needs What materials do you need? Do you need a projector? Do you need internet access?

Workshop planPlease provide a time and activity breakdown for your session. Please state the intended outcomes of each activity.

How can your workshop connect to other workshops?Does this workshop stand alone? Can it connect to other workshops? What parts of your workshop can follow through to another workshop?

Would you like FutureEverything to be present at your workshop?

44

Community Forum on Data, Privacy and Trust (November 2016):

• This workshop increased my own knowledge of data security

• The real impact of CityVerve on Manchester will come from quality data being collected that has a tangible impact on people’s everyday life in the city

Community Forum on the impact of technology on lifestyle (January 2017)

• Through the forum, I learnt about the use and social benefit of tech alongside the specifics of tech software or hardware

• This workshop has been very helpful for thinking about what is actually private and public (in terms of data), and the individual’s role as a citizen

Community Forum on the necessity of technology (February 2017)

• It was great to explore how comfortable I am with how my data might be used/collected. It showed exactly how important communities are in smart cities

• My biggest learning was the insight into smart city options in development - the focus and motivation of smart city development

Community Forum on Accessibility in Future Cities (April 2017)

• This process helps non-digitals learn about smart cities

• My biggest learning was thinking about the use of data and how apps / services can be built wfor specialist groups

Appendix: Comments from event attendees

45

ColophonSeries Editor:Dr. Drew Hemment Published by:FutureEverythingManchester Technology CentreOxford RoadManchesterM1 7EDUKfutureeverything.org2018

Design: Supanaught

The work is licensed under a Creative Commons Attribution-NonCommercial 3.0 Unported License

British Library Cataloguing-in-Publication Data: A British Library CIP Record is available

http://www.futureeverything.org

Evaluation of human-centred design in CityVerve...4. Evaluation of human-centred design in CityVerve...

Documents

Transcript of Evaluation of human-centred design in CityVerve...4. Evaluation of human-centred design in CityVerve...