Research Design

Research Desgin It is the entire design of the research projectIt is said as the research proposal It involves deciding on all aspects of the research

process That includes

Philosophical assumption Research method Data collection techniques Data analysis Publication outlet if possible

It matters to get your proposal accepted or rejected

Research Process

Problem

Liter., Theory

Propositions/ Hypothesis

Research Design

Data Collection & Analysis

Data Interpretation

How

to K

now

ledg

e

Process Iterations

The

ory

Infe

renc

e

Identify Problem & Motivate

Define Problem Show

Importance

Define Objectives of

a Solution

What would a Better Artifact Accomplish?

Design & Development

Artifact

Dis

cipl

inar

yK

now

ledg

e

Met

rics

, Ana

lysi

s K

now

ledg

e

Demonstration Find Suitable

context

Use Artifact to Solve problem

Evaluation

Observe How Effective, efficient

Iterate Back to design

Communica-tion

Scholarly Publications

ProfessionalPublications

Design research process

Step 1: Problems/Questions

A question/problem occurs or is posed to the researcher for which that researcher has no and can find no answer(s).

Example -1 User profiling on the Internet

How can we support an individual user in providing information suited to their needs? There are many approaches to automated profiling of users based on their web habits, and occasionally they throw up some interesting information.

Can we improve how users' information needs are identified, specified, and managed in specific domains, and in general.

Step 2: Literature, Theory, Logic

Available literature/theory is reviewed to determine if there is already a solution to the problem.

Existing solutions do not always explain new conditions.

The existing solution might require some revision or even be discarded.

Example III – Interface Agents

Interface agents have become a technology widely used to provide personalized assistance to users with their computer-based tasks.

Interface agents are computer programs that have the ability to learn a user’s preferences and working habits, and to provide him proactive and reactive assistance in order to increase the user’s productivity.

A commonly used metaphor for understanding interface agent paradigm is comparing them to a human secretary or personal assistant who is collaborating with the user in the same work environment (Maes, 1994).

Example – III Interface agent – Research gap

Horvitz (1999) has pointed out some problems with the use of interface agents: poor guessing about the goals and needs of users, inadequate consideration of the costs and benefits of each agent action, poor timing of agent actions and inadequate attention to opportunities that allow a user to guide the invocation of agent services and to refine potentially suboptimal results provided by the agent.

Research Gapwe consider that interface agent developers have paid

little attention to two key issues when developing agents that assist users: how to best interact with each user and how to provide them assistance of the right sort at the right time.

Hypothesis

We have focused our attention on studying the interaction between a user and an agent, as well as on how a user wants to be assisted. We believe that, in the same way as each person works and interacts in a certain way with his/her personal assistant, each user interacts in a personal way with an interface agent.

Conceptual FrameworkThe proposed solution is by studying user and

agent interaction, we can design a better agent that can serve the user to achieve his/her goals.

Step 3: Propositions/Hypotheses

Application only for Quantitative ResearchThe researcher generates intermediate

propositions and/or hypotheses to gain a better understanding of the problem or describe a solution to the problem.This is at best a temporary solution since

there is as yet no evidence to support either the acceptance or rejection of these propositions/hypotheses.

Hypothesis

Understanding user and agent interaction results in better agent design

Another example Graphical user Interface increases software

learnability Use of Could computing reduces system

interruption Use of Could computing increases organizational

profitability

Step 4: Research Design

Research methodologies reflect high-level approaches to conducting research.The individual steps within the methodology

might vary based on the research problem being performed.

There are three broad approaches to be used in research designQuantitative.Qualitative.Design science

Step 4: Design a Study

Quantitative Qualitative Design science

•Explanation, prediction

•Explanation, description

•Motivated to solve problems

•Test/Modify theories

Build theories Prescriptive – has heuristic nature

•Known variables Unknown variables

Known variables

•Large sample Small sample Small sample, case based

•Standardized instruments

Observations, interviews

Observation, interviews, etc

•Deductive Inductive Inductive

Step 4: Cont’d

Quantitative Qualitative Design science

Statistical generalization

•Analytic generalization

•Replicable to similar problems in similar contexts

•Use numbers Use texts Texts, codes and drawings

•Standard instrument to validation

No standard instrument, triangulation

Pragmatic validity

•Objective Truth Subjective truth Subjective Truth ???

•Linear Process Iterative Iterative

Step 5: Acquire/Analyze Data

The researcher collects data relating to the research problem.The means of data acquisition will

often change based on the type of the research problem.

This might entail only data gathering, but it could also require the creation of new measurement instruments.

Example III – Interface Agents

To prove our hypothesis, we carried out an experiment with real users interacting with different interface agents, and we discovered several user–agent interaction issues that require personalization.

Data collected about:1. we studied what kind of software assistant different

users want2. we studied users’ preferences and assistance

requirements concerning three assistance actions: warnings, suggestions and actions on the user’s behalf.

3. we analyzed users’ reactions towards interruptions, i.e. whether users object to being interrupted by their agents or not

Cont’d

4. we considered users’ tolerance of agents’ errors, since some users are more tolerant than others

5. we investigated how users feel about providing explicit feedback to train their agents. Not all users are willing to give explicit feedback, and their willingness may vary as they interact with the agent

6. we analyzed users’ opinions regarding agents’ ability to perform actions on behalf of users and the possibility of giving agents more autonomy

Step 5: Acquire/Analyze Data

The data that were gathered in the previous step are analyzed as a first step in ascertaining their meaning.

Involves statistical analyses, for the most part.The specific statistical tests and procedures

depend on the nature of the underlying data.

Qualitative researchers used words to understand and describe the phenomena

Step 6: Data Interpretation

The researcher interprets the newly analyzed data and suggests a conclusion.Interpretation means identifying relationship

between variables to draw conclusions Understand the meaning behind the dataThe Data for SW success shows low figures,

below our expectation. What it tells us! Does it mean the SW failed? What causes the Failure? How do we overcome those Failures?

Computer importation increased in Ethiopia. What does it mean?

Example: Data Interpretation -- Interface Agents

As a result of our experiment we found that different users have different preferences and requirements regarding the interaction issues mentioned above. This is an indication that personalization is required if interface agents want to assist users as they expect.

Thus, we propose a definition for a user interaction profile that models a user’s interaction preferences, requirements and habits.

The user interaction profile can be used by an interface agent developers to personalize its interaction with each user and to adapt its behavior to each user’s interaction and assistance requirements.

Step 6: Compare Results to Propositions/Hypotheses

The data will either support the propositions and hypotheses or they won’t.This may lead the researcher to cycle

back to an earlier step in the process and begin again with a new hypothesis.

This is one of the self-correcting mechanisms associated with the scientific method.

Research Proposal

Structure of Research Proposal

1. Title 2. Abstract3. Introduction

1. Research Problem2. Research Objective

4. Literature Review5. Research Method6. Significance of the Study7. Research Budget8. Research Schedule9. Bibliography

Title

It should be concise and descriptive. For example, the phrase, "An investigation of . . ." could be omitted.

Often titles are stated in terms of a functional relationship, because such titles clearly indicate the independent and dependent variables.

Think of an informative but catchy title. An effective title not only pricks the reader's interest, but also predisposes him/her favorably towards the proposal.

E.g. change without change

Abstract

It is a brief summary of approximately 300 words.

It should include the research question, the rationale for the study, the hypothesis (if any), the method and the main findings. Descriptions of the method may include the design, procedures, the sample and any instruments that will be used.

Introduction

The introduction typically begins with a general statement of the problem area, with a focus on a specific research problem, to be followed by the rational or justification for the proposed study. The introduction generally covers the following elements:

1. State the research problem, which is often referred to as the purpose of the study.

2. Provide the context and set the stage for your research question in such a way as to show its necessity and importance.

3. Present the rationale of your proposed study and clearly indicate why it is worth doing.

Cont’d

4. Briefly describe the major issues and sub-problems to be addressed by your research.

5. Identify the key independent and dependent variables of your experiment. Alternatively, specify the phenomenon you want to study.

6. State your hypothesis or theory, if any. For exploratory or phenomenological research, you may not have any hypotheses.

7. Set the delimitation or boundaries of your proposed research in order to provide a clear focus.

8. Provide definitions of key concepts. (This is optional.)

Method – Example of Quantitative

Design -Is it a questionnaire study or a laboratory experiment? What kind of design do you choose?

Subjects or participants - Who will take part in your study ? What kind of sampling procedure do you use?

Instruments - What kind of measuring instruments or questionnaires do you use? Why do you choose them? Are they valid and reliable?

Procedure - How do you plan to carry out your study? What activities are involved? How long does it take?

Design Science

What procedures to follow to design the new artifactFor example for Software Development?Describe how you collect the Requirement? From whom you

want to collect Focused group, interview, questionnaire

How you structure your requirement like class diagram or other tools

What algorithm you want to develop Which language to use for your algorithmWhat you evaluate your design solutions? With which

population group? Why you select that One? How long? Justify all your selection by citing other authors for using

similar method to solve or its appropriateness

Results

Obviously you do not have results at the proposal stage.

However, you need to have some idea about what kind finding you get from your research. Describe the expected research or design outputs

It is used to evaluate your success or failure

Failure to Proposal Writing

Failure to provide the proper context to frame the research question.

Failure to delimit the boundary conditions for your research.

Failure to cite landmark studies. Failure to accurately present the theoretical

and empirical contributions by other researchers.

Failure to stay focused on the research question.

Cont’d

Failure to develop a coherent and persuasive argument for the proposed research.

Too much detail on minor issues, but not enough detail on major issues.

Too much rambling -- going "all over the map" without a clear sense of direction. (The best proposals move forward with ease and grace like a seamless river.)

Too many citation lapses and incorrect references. Too long or too short. Slopping writing.