Usability Test of the Application
-
Upload
ferdie-zamora -
Category
Documents
-
view
3 -
download
2
description
Transcript of Usability Test of the Application
1
1. Introduction
The Wide Area Video Communication Application is a mobile video call
application that offers features namely Video Call, Video Chat, and File Transfer.
1.1 Purpose
The primary purpose of the test was to test the functionality of the application and
investigate whether users are comfortable with the applications features and
functionalities.
Executive Summary
The researchers conducted a usability test to selected students from Cor Jesu
College on November 8, 2014. The students were provided with the application’s
Android Project Kit (APK) and Wireless Fedility (Wi-Fi) connection to test the
functionality of the application.
All the selected students are from the Computer Studies Division (CSD). There
are only 10 selected CSD students to perform the test due to some of the students don’t
have android devices or smart phones.
The test identified some of the problems stated below:
Stability of sessions
The sessions are unstable due to the availability of the web
server. The website where the sessions are made is not always
available. There are times when the website is timed out without
certain reason.
Quality Output
The video and voice quality is sometimes poor due to the
internet connection’s performance. The quality output depends on
how fast and reliable the internet connection is.
Incompatibility of the application to any Android mobile Operating
System (OS).
2
Methodology
Sessions
The researchers recruited 10 CSD students to perform the test. The researcher
gives a brief introduction to what the application is about and the functionality of the
application. Then, the researchers gives proper instruction on how to use the application.
The students were gathered before giving the instructions. The session lasted for
15 minutes including the giving of the instructions and the test questionnaires for
evaluation.
After each task, the administrator asked the participant to rate the interface on a 2-point
Likert Scale with measures ranging from Disagree and Agree. Post-task scenario subjective
measures included (see Attachment B):
the application is easy to use;
the application is modernize and offers friendly user interface;
the application is functional due to new technologies involved namely Android;
and
the application is useful to the society.
After the Post-task scenario subjective measures, the students were asked to rate the
application overall by using a 5-point Likert scale (Strongly Disagree to Strongly Agree) by the
eight subjective measures mentioned below:
ease of use;
frequency of use;
learning ability - How quick would it be for the users to learn the application
and its features;
graphical Appeal – User likes the User Interface of the application;
3
application content – Applications contains sufficient functions for the needs of
the user; and
organized Application.
Lastly, the researchers asked the students the following overall application
questions listed below:
what the participants likes the most in the application?;
what the participants likes the least in the application?; and
comments and Recommendations for improvement.
Refer to Attachment C for the subjective and overall questionnaires.
Participants
The participants were selected students from Computer Studies Division. The
participants completed the test on November 8, 2014 at ITR1 class room. Among the
selected participants, each has an android phone.
Table 1: The table below shows the registration information needed from each users to
login and use the application.
Username Input user’s desired username
Password
Email Address
Session ID and Token
API key
Input user’s desired password
Input user’s valid Email Address
Auto-generated by the server upon
successive registration
Given by the server upon successive
registration
4
Evaluation Tasks/Scenarios
The selected students attempt to complete the eight tasks given by the researchers
listed below:
fill up the registration fields to complete the registration;
after successive registration, Login using the registered accounts;
the application now directs users to the main interface of the application to
provide selection of contacts;
after selecting the desired contact, the application directs users to the feature
options the application provides;
the application offers features namely video call and video chat;
after selecting the desired option, the application will now connect to the other
pair of user to perform the specific feature selected;
wait for the applications response; and
after the application responded, the two pair of users are now connected and ready
to perform the specific feature.
5
Results
Task Completion Success Rate
All the selected students completed all the given tasks. Task Completion
Rates Table is given below.
Table 2: Task Completion Success Rates
Participant Task1
Task2
Task 3
Task 4
Task 5
Task6
Task7
Task8
1
2
3
4
5
6
7
8
Success
Completion Rates
√
√
√
√
√
√
√
√
7
100%
√
√
√
√
√
√
√
√
7
100%
√
√
√
√
√
√
√
√
7
100%
√
√
√
√
√
√
√
√
7
100%
√
√
√
√
√
√
√
√
7
100%
√
√
√
√
√
√
√
√
7
100%
√
√
√
√
√
√
√
√
7
100%
√
√
√
√
√
√
√
√
7
100%
Task Ratings
The selected students completed each tasks given and rated the ease and difficulty
of completing the tasks given the factors listed below:
i was able to do the register and login function with ease and without
further questions;
6
as I go on with the application, I was able to understand each feature and
completed the task given; and
i was able to understand the whole function of the application.
The five-point rating scale ranged from 1 (Strongly disagree) to 5 (Strongly
agree). Agree ratings are combined with the strongly agreed (5) and agreed (4) with a
mean agreement ratings of 5.0 considering that all the participants agrees that they were
able to do the register and login functions without further questions, they were able to
understand each features the application provides and understand the whole functionality
of the application.
Ease of completing the Register and Login requirements
All the selected participants agreed that it was easy to perform the Register and
Login Functions
Understands the provided features the application offers
The selected participants agreed that the features were easy to understand and
they were able to complete the task with ease.
Understands the whole function of the Application
The selected participants agreed that they understand the whole application and
how it works.
7
Table 3: Mean Task Ratings & Agree Percentage
Task Ease of Completing
Register and Login
Understands the
Features
Understands the
whole
Functionality
Overall
1
2
3
4
5
6
7
8
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0 (100%)
5.0
5.0
5.0
5.0
5.0
5.0
5.0
5.0
*Percent Agree (%) = Agree & Strongly Agree Responses combined
Time on Task
The testing of the application and performing the tasks were recorded for each
participant. All the tasks given were completed without difficulties.
The selected participants were given task to fill up the information asked from
register and login, test the features the application offers, and judge if the application was
easy to use and whether they understand the flow of the application. Completion of the
tasks given ranged from 2 minutes minimum time to 10 minutes maximum time.
8
Table 4: Time on Task
P1 P2 P3 P4 P5 P6 P7 P8 Avg. Total
(mins)
Task 1
Task 2
Task 3
Task 4
Task 5
Task 6
Task 7
Task 8
4
5
10
7
3
2
5
4
5
6
5
6
3
3
5
4
5
4
4
7
6
3
5
6
7
4
2
8
7
3
5
7
6
7
3
8
6
5
6
4
4
6
2
7
8
4
5
7
3
9
8
9
9
4
7
6
3
10
8
6
5
4
6
4
4.63
6.38
5.25
7.25
5.88
3.5
5.5
5.25
9
Summary of Data
The table below displays a summary of the test data. Low completion rates and
satisfaction ratings and high errors and time on tasks.
Table 5: Summary of Completion, Errors, Time on Task, Mean Satisfaction
Task Task Completion Errors Time on Task Satisfaction*
1
2
3
4
5
6
7
8
8
8
8
8
8
8
8
8
0
0
0
0
0
0
0
0
4
6
5
7
5
3
5
5
4.7
4.7
4.7
4.7
4.7
4.7
4.7
4.7
* Satisfaction = Mean combined rating across three post-task measures: ease of finding the information, ability to keep track of location in site, and site information prediction accuracy.
Overall Metrics
Overall Ratings
After task session completion, participants rated the application for eight
overall measures. These measures include:
10
Table 6: Post-Task Overall Questionnaire
Strongly
Disagree
Disagree Neutral Agree Strongly
Agree
Mean
Rating
Percent
Agree
Thought
application was
easy to use
0 0 0 0 8 3.9 100%
Would use
application
frequently
0 0 4 2 2 3.9 92%
Found it fast
and easy to
learn the
application and
its features
0 0 0 0 8 3.9 100%
Graphical User
Interface makes
me want to
explore more of
the application
0 1 4 1 2 3.9 85%
Application
content would
keep me using it
0 0 4 3 1 3.9 90%
Application is
well organized
0 0 2 5 1 3.9 95%
Likes and Dislikes
Upon completion of all the tasks, selected participants were asked about their
comments and provided the researchers feedback of what they like the most and what
they like the least in the application.
11
Liked Most
The participants were asked what they liked the most and give the following
replies:
i like doing the video call and voice call feature;
i liked graphical output of the video call and chat feature;
Liked Least
The participants were asked what they liked the least and give the following
replies:
it takes several seconds to display the video stream;
Conclusion
Most of the selected participants found that the application is very useful to the
school and to the work groups within the society which lessen their time contacting
others specially when it’s an emergency due to the application is available anytime as
long provided with fast and reliable internet connection to process the video feeding.
12
ATTACHMENTS
Attachment A – Background Questionnaire
Name: _____________________________________________________
School: ____________________________________________________
Age: ______
Gender: __ Male
__ Female
Status: __ Single
__ Married
__ Separated
__ Widowed
Occupation: _________________
Attachment B – Post-Task Questionnaire
Directions: In answering the questions below, use this scale (From standard form for evaluation):
5 = Strongly Agree/Always 4 = Agree 3 = Neutral/Often
2 = Disagree 1 = strongly Disagree/Never
13
For each question, please circle the number that best represents your opinion.
Table 7. Post-task scenario subjective measures
A. Post-task scenario subjective measures
The application is easy to use. 5 4 3 2 1
The application is modernize and offers friendly
user interface.5 4 3 2 1
The application is functional due to new
technologies involved namely Android.5 4 3 2 1
The application is useful to the society. 5 4 3 2 1
Table 8. Task Ratings
Task Ratings
It was easy to do the register and login
function.5 4 3 2 1
The features are easy to use and understand.
5 4 3 2 1
The application’s whole functionality is very
understandable.5 4 3 2 1
14
Attachment C
Table 9. Post-session Overall Subjective Questionnaire
Overall Application Questions:
What you liked most?
What you liked least?
Any recommendations for
improvement of the application?
Attachment D – Task Scenarios
Directions: In answering the questions below, use this scale (From standard form for evaluation):
5 = Strongly Agree/Always 4 = Agree 3 = Neutral/Often
2 = Disagree 1 = strongly Disagree/Never
15
For each question, please circle the number that best represents your opinion.
Table 10. Post-Task Overall Questionnaire
Post-Task Overall Questionnaire
1. Thought application was easy to use 5 4 3 2 1
2. Would use application frequently 5 4 3 2 13. Found it fast and easy to learn the application and
its features.5 4 3 2 1
4. Graphical User Interface makes me want to explore
more of the application5 4 3 2 1
5. Application content would keep me using it 5 4 3 2 16. Application is well organized 5 4 3 2 1