Usability test report roomie

Last update: 2014.3.18

Zhao Wenbin Chen Cheng Wu Tianyi Xu Yihua Zhu Ruoyu Zuo Xingyuan

ROOMIE Usability Test Report

1

Content 1.

Introduction About

Roomie Target

population of Roomie

Goal of usability testing

2. Methodology Overview

Participant Selection & Recruitment

Test Protocol Test

Environment and

Equipment

3. Results Score analysis

Test 1 Findings

Test 2 Findings

4. Conclusion Conclusion

5. Appendix Axure Propotype

Usability test viedo

2

About Roomie Roomie is an online pla,orm that aims at helping university students find suitable roommates in Hong Kong according to their personality, life style and other daily issues.

3 1.Introduction

Target popula4on of Roomie

While the primary users of Roomie are students from Mainland China, as most of them cannot apply for the university dormitories, we also prefer to target graduates who determine to work in Hong Kong but lack roommates to share an apartment, as well as local students who prefer to live separately from parents.

Goal of usability tes4ng

Our goal for the usability tesCng was to collect both quanCtaCve and qualitaCve data of new users learning how to use Roomie to search for potenCal roommates for the first Cme. We hope that the results of the usability tesCng can be used by us to understand the strengths and weaknesses of this project and serve as a benchmark for future usability tesCng.

4 1.Introduction

Overview Usability tesCng occurred during the one-‐week span of March 9-‐16. We divided our tesCng into two rounds, the first round on March 9 and the second round on March 16. ALer the first round tesCng, we opCmized our prototype according to parCcipants’ suggesCons, then held the second round tesCng one week aLer. Ten parCcipants were recruited who had never heard of our project before but yet had a need to look for roommates, or had related roommate-‐finding experiences. During the half-‐an-‐hour tesCng sessions, each parCcipant was asked to do the task of searching for suitable roommates on Roomie. Time spent on the task and qualitaCve comments were collected for each parCcipant. Each session was recorded using QuickTime to allow for later analysis.

5 2. Methodology

Prototype Building

We used Axure to make the prototype of Roomie online platform. Axure is a business drawing software for creating mockups and wireframes of application user interfaces. This easy to use software generates HTML prototypes and functional specifications.

6 2. Methodology

Par4cipant Selec4on & Recruitment

Since our goal was to find the strengths and weaknesses of this project, we recruited users who had never heard Roomie before but had a need to look for roommates, or had related roommate-‐finding experiences. They should be future students, present students or graduates of a certain university in Hong Kong. We recruited our parCcipants through our social networks, and offered them a small giL as an incenCve for parCcipaCon.

7 2. Methodology

Test Protocol

Each test session last for one hour, including 10 minutes for a pre-‐test background interview and post-‐test debriefing quesCons, and 20 minutes for the tasks.

We began each test session by asking them some general quesCons about how they got along with their former roommates, whether they had some good or bad experiences, and what they cared the most when looking for a roommate. Then we gave them two tasks, and asked them to complete each one while talking aloud about their thought process and any confusion they encountered.

8 2. Methodology

Test Protocol The two tasks we gave them were organized by the sequenCal order. The summary of tasks is listed below.

Task Number Task Summary

1 Register for Roomie

2 Search for roommates on Roomie

9 2. Methodology

ALer the parCcipant completed the tasks, we conducted a post-‐test brief, asking them about their interesCng or confused points about Roomie. We also asked them to score each page according to their saCsfacCon (4-‐point Likert scale, from 1 =dissaCsfied to 4=saCsfied) and asked if they thought they would be likely to use Roomie, based on their iniCal impression.

Test Environment and Equipment

The tests were conducted in Learning Commons of Wu Ho Man Yuen Building. We used QuickTime to record our test screens. We each took turns moderaCng one session and observing one session. We set up QuickTime on a Mac, and give this Mac to parCcipant to use during the test.

10 2. Methodology

Data Collec4on

We collected both qualitaCve and quanCtaCve data about the parCcipants’ behavior. Our quanCtaCve data consisted of the performance data of Cme spent on tasks for each parCcipant, and their saCsfacCon (4-‐point Likert scale) towards Roomie. The qualitaCve data we collected were quotes, comments and observaCons noted down by the observer during the test.

Score analysis：Test 1*

3.33.6 3.6

2.5

3 3.12.8

0

0.5

1

1.5

2

2.5

3

3.5

4

登陆注册页

邮箱注册

填写信息

个人主页

首页

他人的个人主页

加好友

Tes t 1均分

Test1均分

11 3.Results

*Test 1 stands for the first-round test. After the first round, we optimized our prototype and conducted Test 2, which is the second–round test.

Score analysis: Test 2

2.7

2.8

2.9

3

3.1

3.2

3.3

3.4

3.5

登陆注册页

邮箱注册

填信息页

个人主页

首页

搜索结果页


感兴趣

留言页

Tes t 2均分

Test2均分

12 3.Results

Score analysis: Test 1 & Test 2

0

0.5

1

1.5

2

2.5

3

3.5

4

登陆注册页

邮箱注册

填写信息

个人主页

首页


test1均分

test2均分

13 3.Results

Time spent on task: Test 1 & Test 2

T ime spent on task

0

5

10

15

20

25

1 2 3 4 5

Minutes Test1

Test2

14 3.Results

From this chart we can see clearly that aLer opCmizing the prototype, parCcipants spent less Cme on compleCng tasks than the first Cme. It means that now our prototype is easier to understand, and the searching process is clearer than before.

Findings

In the first-‐round test, we found 5 broken experiences and figured out soluCons for each broken experience. In the second-‐round test, we found 8 problems that needed to be solved.

15 3.Results

Prototype V1.0

Prototype V2.0

Prototype V3.0

Fixed 5 problems from Test 1


3 person points out that the picture on the first page is not appropriate. “The picture in registraCon page cannot clearly represent the funcCon of Roomie.” “The picture make the website look like a social media.”

16 3.Results Test 1

Broken experience 1

SoluCon 1 Changed to a more clear and high quality picture, cannot find a suitable picture with Asian faces.

17 3.Results Test 1

Broken experience 2 5 users menConed that the design of user’s own informaCon page is not clean and clear. “I find the table very stressful and the classificaCon of informaCon is not clear. ”

18 3.Results Test 1

SoluCon 2 Change table to listed informaCon, add more sucCons in user’s own informaCon page.

19 3.Results Test 1

3 person points out that “The advanced search is confusing”, “ I do not understand the relaConship between apartment and roommate.”

20 3.Results Test 1

Broken experience 3

SoluCon 3 Change boxes to listed opCons to make it easier to understand, and also add missed bufons.

21 3.Results Test 1

4 users menConed that more personal informaCon such as lifestyle is necessary. “I hope to see more personalized informaCon to know what kind of person the potenCal roommate is.”

22 3.Results Test 1

Broken experience 4

SoluCon 4 Provide choices related to lifestyle and apartment requirements .

23 3.Results Test 1

3 person points out that the whole process of registraCon and search is not very smooth, some bufons are missing. “I think there should be a next or finish bufon.” “ I can’t go back to the homepage.”

24 3.Results Test 1

Broken experience 5

SoluCon 5 Adding navigaCon at the top of each page, and change “submit” to “finish”.

25 3.Results Test 1

After we fixed problems from Test 1, we updated our prototype from V1.0 to V2.0 and conducted Test 2.

26

Prototype V1.0

Prototype V2.0


In the “fill-‐in personal informaCon” page, we want to collect more informaCon about the user. But 3 users complained that they don’t want to fill in all the informaCon. “Some informaCon I don’t want to share to others. It’s all required? ”

27 3.Results Test 2

Broken experience 1

We add “*” in front of those mandatory fields to disCnguish from opConal ones, so as users can clearly figure out which part they must fill in.

28 3.Results Test 2

SoluCon 1

2 users pointed out that they don’t want to type in the descripCons of their characters, or it took some Cme to think about how to describe themselves. So there is a high possibility that they will leave in this stage. “I don’t want to type, I want to choose.”

29 3.Results Test 2

Broken experience 2

Besides words to describe the characterisCc,we first provide menu for users to choose their character type.

30 3.Results Test 2

SoluCon 2

31

2 person points out that they can not understand the word ‘洋楼’and‘唐楼’, and actually the type of the building is not that important. Moreover, the type of the apartment is something that they concern more. “I don’t know how to choose, I have no idea what it is.”

3.Results Test 2

Broken experience 3

To be more clear, we change “唐楼” and “洋楼” to “无电梯楼宇” and “有电梯楼宇”. We also add Type of Apartment to provide more details.

32 3.Results Test 2

SoluCon 3

ALer registraCon process, user will landing to personal homepage, but there is no instrucCons for what to do next. “OK, I’ve registered. So what I goanna do next? ”

33 3.Results Test 2

Broken experience 4

SoluCon 4 Besides words to describe the characterisCc,we first provide menu for users to choose their character type.

34 3.Results Test 2

RecommendaCon funcCon on the homepage is overlooked by users. “See I can search the website. If there are recommended roommates for me, that would be great!”

35 3.Results Test 2

Broken experience 5

SoluCon 5 We put recommendaCon part on the top of homepage to reinforce its funcCon, and put search bar on the right side. If users want to search more informaCon, they can click Advanced Search and go to search page.

36 3.Results Test 2

37

2 person points out that when searching, they put gender in priority. “Where is the gender in the search box? I would definitely want that opCon before all the other choices. ”

NO GENDER!

3.Results Test 2

Broken experience 6

We rearrange the choices and add “同一性别” so as it can be seen more easily.

38 3.Results Test 2

SoluCon 6

In the process of finding search funcCon, user click “people I interested in” below head portrait instead of “search” bufon on the header. However, “people I interested in” is not the right search bufon. There is some misunderstanding of “people I interested in”. “I’ll click this(‘people I interested in’) to see who is my suitable roommate.

39 3.Results Test 2

Broken experience 7

We build list page of users who I am interested in aLer click “people I interested in ” bufon. Users can check and manage all the list here.

40 3.Results Test 2

SoluCon 7

41

2 person points out that they can’t understand what grouping means when I want to leave a note to other users. “I don’t know what the grouping here means, will it appear every Cme I leave message for someone? Will other people see my grouping informaCon? Would it be more appropriate if it shows when I liked someone?”

3.Results Test 2

Broken experience 8

We revise Grouping facCon, add a drop list for users to select and add group category.

42 3.Results Test 2

SoluCon 8

Further demand of renCng a house can not be fulfilled in our website “Besides finding a roommate, I want to find a flat. If there is only informaCon of roommates, I think that’s not enough.”

SoluCon Unfortunately,we cannot solve this problem so far.

43 3.Results Test 2

Broken experience 9

The tasks are about the basic funcCons of Roomie. Most users successfully finished all the tasks. All of them experienced some difficulCes but all successfully figured out a way to overcome them.

From the first-‐round test we found many weaknesses of Roomie and opCmized it according to parCcipants’ suggesCons. And in the second-‐round test, though Roomie had been opCmized, parCcipants sCll figured out some broken experiences that need to be solved. It means that to saCsfied users’ real needs, we sCll have a lot to do.

44 4. Conclusion

• Roomie usability test video

• Prototype (in the folder “V3.0/ index.html”)

45 5. Appendix

Thank you

46

Usability test report roomie

Technology

Transcript of Usability test report roomie