Planning Research Strategies for the QR Cache Research...

Running Head: PLANNING RESEARCH STRATEGIES 1

Planning Research Strategies for the QR Cache Research Project

Robert Power

Athabasca University

PLANNING RESEARCH STRATEGIES 2

Table of Contents

Abstract ......................................................................................................................................................... 3

Planning Research Strategies for the QR Cache Research Project ............................................................... 4

Choosing a Research Paradigm ..................................................................................................................... 4

Proposed Research Questions .................................................................................................................. 5

Research Methods Alternatives ................................................................................................................ 6

Data Collection and Analysis ......................................................................................................................... 9

Data Collection Options ............................................................................................................................ 9

Proposed Data Collection Strategy ......................................................................................................... 13

Data Analysis Procedures ........................................................................................................................ 14

Additional Practical, Ethical and Philosophical Issues ............................................................................ 16

Conclusions ................................................................................................................................................. 19

References .................................................................................................................................................. 20


Abstract

High mobile device penetration, universal connectivity, and the expressed needs of stakeholders

combine to create an opportunity to examine the effectiveness of mobile learning strategies in Qatar

(MacLeod, 2011; Metodieava, 2012; Nagy, 2012; Warraich & Dahlstrom, 2012). The QR Cache research

project aims to examine the effectiveness of mobile reusable learning objects (RLOs) accessed by

scanning Quick Response codes at enabling students to learn English computer terminology through

situated learning using their own mobile devices. It also aims to provide an exemplar, demonstrating

how such RLOs could be created and deployed to effectively meet training needs. This paper examines

the selection of an appropriate research paradigm and questions for the QR Cache project. Alternative

research methodologies are presented, along with an overview of appropriate data collection and

analysis strategies. Issues related to the practical, ethical and philosophical implications of the research

design are also discussed.

Keywords: data collection, data analysis, Design-Based Research, mixed-methods research, mobile

learning, paradigms, pragmatism, Qatar, QR Codes, research methodologies, research questions,

Reusable Learning Objects,


Planning Research Strategies for the QR Cache Research Project

QR Cache: Assessing the feasibility of using Quick Response (QR) codes and mobile devices to

help students master computer-related workplace English competencies (Power, 2012c) is a research

project being undertaken at College of the North Atlantic-Qatar (CNA-Q) to explore how mobile reusable

learning objects (RLOs) could be used to meet expressed stakeholder needs. A detailed literature review

of the theoretical foundations of m-learning pedagogy and the design of effective mobile RLOs is beyond

the scope of this paper. This paper aims to explore the specific research paradigm and questions for the

QR Cache project. Paradigmatic alternatives and potential research questions for this project have been

explored in a previous paper (Power, 2012b). A conceptual framework (Power, 2012a) was developed

to help focus on the most appropriate research paradigm and questions by mapping the specific

variables under examination, and their relationships in the local context (Cohen, Manion & Morrison,

2011, p. 117; MacDonald, Stodel, Muirhead & Thompson, n.d.). After establishing the research

paradigm and questions, this paper examines alternative research methods, and the specific data

collection and analysis strategies that would be needed to implement one methodological alternative.

This includes an examination of practical and ethical implications, as well as philosophical issues related

to the research methodology.

Choosing a Research Paradigm

A pragmatic approach has been previously examined as an appropriate paradigm for the QR

Cache project (Power, 2012b). It is compatible with the theoretical foundations of m-learning design, as

well as the project’s ontological and epistemological positions. It is also compatible with the aim of

providing answers to real, practical questions, and results that will be of benefit to learners and

practitioners (Cohen et al., 2011, p 23). A nominalist ontological perspective has been established for

the QR Cache project because of its alignment with the theoretical foundations of Activity Theory (AT)

(Kaptelinin & Nardi, 2006), Transactional Distance Theory (TDT) (Moore, 1989, 1991) and Koole’s (2009)

FRAME Model. These theoretical bases

focus on interactions between learners, other individuals, and content and technology [so an]

ontological perspective that focuses on individual and group creation of social reality would

seem to have more utility… (Power, 2012b, p. 6).

Such ontological positioning requires an epistemological approach that focuses on qualitative methods

of acquiring and communicating knowledge (Cohen et al., 2011, p. 6), or that integrates appropriate

quantitative and qualitative methods. Pragmatism is an appropriate paradigm because it acknowledges

multiple facets to truth and reality, and focuses on the utility of the research findings in the practical

world. It is described as a practice-driven approach to research, and as methodologically eclectic. It

draws upon whatever combination of specific approaches (whether quantitative, qualitative, or both) is

best suited to the context-dependent needs. However, Denscombe (2008) cautions that such

eclecticism is driven by the research questions themselves, and


is not an ‘anything goes’, sloppy, unprincipled approach; it has its own standards of rigour, and

these are that the research must answer the research questions and ‘deliver’ useful answers to

questions put by the research (as cited in Cohen et al., 2011, p. 23).

The compatibility of pragmatic research with a variety of strategies is also desirable because of the

utility of mixed-methods research for triangulation of findings (Denzin, 2008) and collection of different,

yet equally useful types of data (Cohen et al., 2011, p. 23). In addition, an eclectic mix of theoretical and

methodological approaches may also facilitate discourse across paradigmatic boundaries, thereby

increasing the overall impact upon educational practice and future research (Newbury, 2011). This

decision is congruent with the refined research focus resulting from the development of a conceptual

framework for the QR Cache project (Power, 2012a). That framework illustrates the interrelationships

between the local technological context, explicitly stated stakeholder needs, and the theoretical

foundations of effective m-learning design. It also identifies the responses of students and teachers to

the use of the mobile RLOs as gaps in the current understanding of the effectiveness of such RLOs at

meeting local stakeholder needs. Increasing understanding of how students and teachers interact with

learning content, and each other, represents information that will be of immediate (pragmatic) utility to

local stakeholders. It is also the foundation of the research questions, which guide the subsequent

selection of methodological approaches, data collection options, and the ultimate analysis of the data

collected.

Proposed Research Questions

Potential pragmatic research questions for the QR Cache project were explored in a previous

paper, along with ontological and epistemological positioning (Power, 2012b). The development of a

conceptual framework (Power, 2012a) has enabled the articulation of a more appropriate and focused

set of questions that are consistent with pragmatic, mixed-methods research. The refined research

questions are articulated below:

1. How do learners respond to the use of mobile RLOs, accessed by scanning QR codes, to learn

English computer terminology and concepts?

a. Do learners experience any difficulties when accessing the RLOs?

b. Do learners enjoy using such mobile RLOs?

c. Would learners like to use such RLOs more often?

d. Is there any impact on student achievement?

2. How do teachers respond to the integration of mobile RLOs, accessed by scanning QR codes, to

teach English computer terminology and concepts?

a. Do teachers experience any difficulties when accessing the RLOs?

b. Do teachers enjoy using such mobile RLOs?

c. Would teachers use similar RLOs to teach other topics/courses?

3. Does the integration of the mobile RLOs, accessed by scanning QR codes, reflect the principles

and benefits of effective m-learning design?

a. Do the RLOs help to reduce transactional distance between learners and content,

learners and other learners, or learners and teachers (Moore, 1989, 1991)?


b. Do the RLOs create optimal interaction between individuals, technology, and social

elements, as outlined by the FRAME model (Kaptelinin & Nardi, 2006; Koole, 2009)?

Research Methods Alternatives

Pragmatic research allows for a mixture of divergent methodologies as appropriate to fulfill the

overall aims of a research project, and to provide answers to specific research questions. A mixture of

both quantitative and qualitative methods could be used for the QR Cache project. As currently

articulated, only one specific sub-question (Is there any impact on student achievement?) would require

a purely quantitative (experimental) methodological approach. The remaining research questions, and

their theoretical bases, are intended to demonstrate how the use of such mobile RLOs can create

powerful new learning opportunities (Kozma, 1994a, 1994b). These questions are also likely to be the

central focus of future iterations of the QR Cache project. These questions focus on how learners and

teachers interact with the content, the technology, and each other, when using the mobile RLOs.

Elucidation of these issues could be achieved by using different approaches to gathering both

quantitative and qualitative data. One approach would be case study research. Another approach

would be a mixed-methodological study based upon the principles of design-based research (DBR).

A case study approach would be useful because it could provide a wealth of easily

contextualizable details for practicing teachers (Cohen et al., 2011, p. 289). Case studies provide

narrative accounts of data collected through a variety of means, including observations, interviews, and

surveys. The results are presented as rich, thick descriptions that allow the reader to see the intricate

details of what is happening. They also allow the reader to easily generalize the results to their own

contexts. Such an approach would be consistent with the pragmatic framework of the QR Cache project

because practicing teachers could readily see how the mobile RLOs have affected learners, and how the

benefits of the RLOs could be transferred to their own classroom and subject area contexts (p. 289).

Thus, using a case study approach would achieve the objective of providing a mobile learning exemplar

for other teachers.

Deciding to use a case study approach requires a careful weighing of both practical and

philosophical elements. One practical benefit of case study research is its focus on a small, closed group

(or unit) of participants (Gerring, 2004). The QR Cache project aims to study the use of the RLOs by a

relatively small group of students enrolled in a specific course for a period of approximately eight weeks,

so it certainly meets the criteria for a case study participant group. However, the limited student

sample size may present limitations to the amount of data on learner perspectives that could be

collected, so this approach might be more suitable if the project were expanded to encompass

additional class sections or courses (Cohen et al., 2011, p. 291). The limited time frame of the course

also presents practical problems for a case study. One problem is the practicality of conducting either

whole group observations, or participant interviews. Case studies typically involve observations

spanning extended periods of time (Cohen et al., 2011, p. 298). Conducting participant interviews is also

a time-consuming endeavor, which is further complicated by the already tight class schedules of both

the target participants and the researcher. In addition, the RLOs themselves present a constraint upon

data collection for a case study. The RLOs cover a single course module spanning two one-hour class


periods. This presents the problem of minimal classroom observation time, and would likely result in

too little data being gathered through this approach to be of benefit to the aims of the research (p. 459).

From a philosophical perspective, there are a number of issues which must be considered

before deciding to use a case study approach. The first issue is the role of the researcher. It would be

difficult for the researcher, in this scenario, to act exclusively as an outside observer (Cohen et al., 2011,

pp. 297-298; Dwyer & Buckle, 2009). This is due to the fact that the researcher is currently acting as a

classroom teacher for the target course, and has also acted as the instructional designer for the RLOs. It

would be possible to act as a participant observer, which is a particularly desirable option, as one of the

aims of the integration of the RLOs is to reduce learner-teacher transactional distance by increasing

interaction (Koole, 2009; Moore, 1989, 1991). However, this raises issues of researcher bias. Flyvbjerg

(2004) argues that case studies actually permit a great deal of reflexivity, and a tendency towards

falsification of researcher bias. However, close teacher-student interaction and classroom time

constraints may result in a constrained observational focus on pre-ordinate variables. Participant

observation “risks being highly selective, unrepresentative and more concerned with the agenda of the

researcher rather than the real situation” (Cohen et al., 2011, p. 468). The researcher may miss intricate

and important details of classroom interactions that could force a re-evaluation of research hypotheses.

Related to the role of the teacher as a researcher are power issues. Participant students may

perceive a power imbalance which could distort observed data. Such a power imbalance may cause

students to behave differently, and to interact differently with the RLOs, than they would under normal

circumstances. However, it could be argued that this is both a natural consequence of being a student

in a school, and a desired effect of pairing students with a teacher (Rost, 2006). One role of a teacher is

to generate interaction with content and learners, and to generate enthusiasm for the learner activities

that are taking place. Considering this, the presence of a power imbalance could be viewed as a

necessary variable in the research, and could be analyzed from the perspective of using technology as a

bridge between learners, teachers and content (Koole, 2009; Moore, 1989, 1991). Conversely, power

imbalance may have an adverse effect upon understanding students’ perspectives about the use of the

RLOs. Case study research has the ability to give participants a strong voice. Detailed observations of

participant behavior and commentaries provided in participants’ own words allow readers to see and

hear the reality of participant responses with minimal filtering and interpretation. However, a power

imbalance stemming from the participant-researcher’s role as a classroom teacher may cause students

to provide commentaries that they believe the teacher will find pleasing (Cohen et al., 2011, pp. 204,

422). This is especially true in the Qatari context, where there is a cultural tendency to avoid conflict

and save face (Clearview, n.d.; Qatar Academy, n.d.).

While a case study approach would be consistent with the pragmatic paradigm of the QR Cache

project, its practical and philosophical limitations force the consideration of an alternative

methodological approach. A pragmatic, mixed-methodological approach would be an appropriate

choice in this context. However, the specific mixture of methodologies cannot be a haphazard affair

(Cohen et al., 2011, p. 23). Their selection, along with data collection and analysis strategies, must be

guided by the intended purpose of the research—answering the specific research questions. Design-

based research (DBR) is an appropriate methodological approach in light of the context of the QR Cache


project. DBR is inherently pragmatic, it aims to implement and investigate effective educational

solutions across multiple iterations, and it is linked to understanding and expanding theoretical

foundations (Anderson & Shattuck, 2012; Design-Based Research Collective [DBRC], 2003). The DBRC

(2003) describes DBR as a methodology displaying five key characteristics. DBR is research that focuses

on “designing learning environments and developing theories” (p. 5). DBR also flows through a series of

cycles, or iterations, in which designs and theoretical assumptions are revised to fit observed

circumstances. DBR research must “lead to sharable theories that help communicate relevant

implications” to other practitioners and educational designers (p. 5). DBR design must focus on

application in real settings. Finally, DBR must employ strategies that can actually “document and

connect processes of enactment to outcomes of interest” (p. 5). Since local interest has been expressed

in expanding upon the QR Cache project (and integrating the RLO strategies into courses offered by

other departments), and since the RLOs are intended to be a demonstration of interventions with

multiple positive effects, DBR does present a suitable framework for guiding the methodological

approaches for this research.

DBR also allows for the integration of a variety of methodological tools, as appropriate for

answering specific research questions (Anderson & Shattuck, 2012). From a practical perspective, DBR

iterations can be designed (as the name design-based research suggests) to meet local needs and

contexts. In the QR Cache project, the current iteration could be regarded as the first phase of DBR, in

which the intervention is pilot tested with a smaller participant base and content focus. Future

iterations could “evolve through… the testing of prototypes, iterative refinement, and continuous

evolution of the design” (p. 17) in response to emerging understandings of stakeholder responses and

specific needs. Multiple iterations could be used to guide the evolution of a mobile RLO strategy that

could be implemented campus-wide (and potentially even be adopted in the workplace by the

employers who sponsor CNA-Q’s students). This sequence of design evolution and feedback could

ultimately be used to expand understandings of the theoretical and practical foundations of m-learning

design. Of course, practical limitations to the use of a DBR design can be seen in such factors as funding

and stakeholder response. Many DBR projects fail to progress through the entire cycle of iterations

because “the iterative nature can exceed the resources or time available to researchers or funding

bodies” (p. 21). Even in situations where research funding may be available, approval to release such

funds may be contingent upon perceptions of the utility of the research. This utility is something that

would need to be demonstrated through stakeholder responses to each iteration of the research.

From a philosophical perspective, DBR’s inherent pragmatism is attractive. DBR allows for the

use of appropriate tools and techniques to gather and analyze relevant data, and to mitigate

philosophical and ethical concerns. One philosophical concern with DBR that has been raised by

adherents of positivist, quantitative research is that it does not prescribe easily generalizable results to

specific, discrete questions (DBRC, 2003). However, proponents of DBR argue that this is a

methodological strong point. They argue that it is difficult to draw generalizations from discrete,

isolated variables that generate a true understanding of the complexities of teaching and learning

practice (Anderson & Shattuck, 2012; DBRC, 2003). Even if the results of a particular DBR study are not

readily generalizable to other institutional or subject area contexts, they can produce a positive impact


upon teaching and learning practices in the local context. As with a case study approach, there are

philosophical issues that would need to be addressed with respect to the role of the researcher, power

issues, and participant voice. However, these issues may be more easily addressed through the specific

mixture of methodological tools permitted within DBR. For instance, detailed on-site observations

would not be needed, eliminating the effects of researcher bias in the types of observations recorded. A

mixture of quantitative and qualitative data could be collected by the use of surveys, instead of

observations or interviews. This could mitigate the effects of a power imbalance between the teacher-

researcher and students, give students more opportunity to freely express their own voice, and provide

a means of triangulating quantified data with qualitative expositions (Cohen et al., 2011, p. 382). The

role of the researcher as a participant teacher must also be carefully considered when adopting a DBR

approach. DBR typically involves a synergistic partnership between practitioners and researchers,

allowing each to focus on their respective roles and specializations (Anderson & Shattuck, 2012). While

the researcher is also teaching a section of the target course, that section is not participating in the RLO

intervention. Future DBR iterations would benefit if the researcher were to completely recuse himself

from also serving as a participant teacher (i.e., if the iteration were to be studied in the context of

courses from a different department or subject area).

Both the case study and DBR approaches have appeal in the context of the QR Cache project.

However, DBR has particular appeal because of its fit with the design of the current iteration of the

research at CNA-Q, and its ability to guide the use of the current research results, and the development

of future iterations of the project. The remaining sections of this paper will focus on the current QR

Cache project as the first iteration of pragmatic research guided by DBR methodology.

Data Collection and Analysis

The research questions articulated for the QR Cache project require a variety of data collection

and analysis methods. Some of the questions will require quantitative (or at least quantifiable) data,

while others will require data of a more qualitative nature. A range of options could be considered as

both consistent with a pragmatic framework, and the answers needed to the specific research

questions. The major options are examined below, along with discussion of practical, ethical and

philosophical considerations. This is followed by a closer look at the instruments and a data collection

strategy, proposed data analysis procedures, and specific practical, ethical and philosophical

considerations.

Data Collection Options

One data collection option is the use of tests to collect data on student achievement. Tests can

provide quantitative data that could be statistically analyzed to determine if there are significant

differences between the achievement of students who use the mobile RLOs, and those who are

instructed using the existing course workbook (Cohen et al., 2011, p. 478). A number of items must be

considered when using tests as a data collection strategy. The first is the type of comparison needed.

Tests can be either criterion-referenced (measuring mastery of specific skills or concepts), or norm-

referenced (providing an achievement score in relation to that of other students). Careful attention


must also be paid to the crafting of test questions to ensure their validity (that they measure the skills

they are explicitly intended to measure) and reliability (that the measurement they provide is consistent

across test applications) (pp. 483-484).

The target course (and unit of study) for the QR Cache project uses competency-based

assessment strategies, so it is best described as criterion-referenced. Assessment uses standardized

Practical Evaluations (PEs) and Knowledge-Based Tests (KBTs). The construction of these tests is

governed by guidelines from the Australian Technical and Further Education (TAFE) organization, with

complete mastery (a score of 100 percent) required to pass a PE, and a minimum score of 70 percent

required to declare competency from a KBT. Since the assessment construction and evaluation is

governed by stringent guidelines, it is not possible to manipulate the test items. However, the strict

TAFE guidelines and assessment approval process does provide a degree of confidence in both the

validity and reliability of the instruments (Cohen et al., 2011, pp. 479-480). Also, the use of the

standardized tests allows for easy, valid and reliable comparison of achievement results between

participants in the QR Cache project, control group participants, and previous administrations of the PEs

and KBTs. Such statistical analysis is further aided by the nature of the test items themselves. The PEs

and KBTs assess basic terminology and factual recall, so the test items consist primarily of multiple

choice, matching, and fill-in type questions. These types of questions (and student responses) are easily

coded, and can be easily transcribed into statistical analysis software packages such as Statistics

Program for the Social Sciences (SPSS) (p. 382).

While the primary aim of the QR Cache project is to elucidate student and teacher responses to

the use of the RLOs, the examination of achievement levels would be useful in providing practicing

teachers with a degree of confidence in their effectiveness compared to other teaching methods. Since

the PEs and KBTs are already in use, analysis of the data does not represent a significant increase in

either time or resource commitments. The minimal additional burden, combined with the potential

utility of the data to readers of the research report, makes their use an attractive data collection option.

Online surveys are another attractive option. Surveys allow for the collection of both

quantitative and qualitative data, and are a useful tool for gaining feedback from large numbers of

participants in relatively short periods of time (Cohen et al., 2011, p. 256). In this context, web-based

questionnaires would be developed for two target groups—student and teachers. These can include a

range of fixed-response questions such as demographic information, Yes/No responses, multiple choice

questions, or ranking or rating (Likert) scales (p. 382). However, while these types of questions elicit

easily codified and statistically analyzed responses, they do pose limitations to the gathering of relevant

qualitative information. This can be mitigated by including open-response questions, which provide

participants with an opportunity to expand upon their responses to fixed-response questions.

When designing the questionnaires, careful attention must be paid to a number of factors. One

factor of relevance is participant literacy (Cohen et al., 2011, p. 398). All of the learner participants are

English Foreign Language (EFL) students. Due to minimum EFL competency requirements for

progression through their program of study, it can safely be assumed that the students would have

adequate skills to accurately respond to simply-worded fixed-response questions. Question wording


must be carefully considered so as to avoid unnecessary confusion, as the EFL participants may have

difficulty with the interpretation of negative statements, or culturally contextualized phrases (p. 379).

The wording of instructions must also be kept simple, and easy to follow. Instructions should be broken

up throughout the questionnaire, so that the most relevant instructions are easy to find and follow at

any given time. It is also advisable to avoid excessive use of filtering responses in the current context, as

this may create undue confusion. Respondents who misunderstand the filtering queues may end up

completing all sections of the questionnaire, even if they do not need to.

The incorporation of scale-response questions, such as Likert scales, presents a challenge

(Cohen et al., 2011, pp. 387-390; Trochim, 2006). While they are easy to answer, care must be taken to

minimize the central tendency effect (the tendency of respondents to avoid selecting responses at

either extreme of the rating scale, thus effectively reducing the practical size of the scale) (Cohen et al.,

2011, p. 388). English competency levels may have an impact upon response rates to open-response

questions. Care must be taken to limit the number of open-response questions to avoid participant

fatigue, and to maximize overall survey completion rates (pp. 277-279, 382). Care must also be taken in

the placement of open-response questions. Interspersing such questions amongst a higher volume of

fixed-response items would prevent the questionnaire from appearing either too long or daunting,

thereby increasing completion rates.

Questionnaires for this project should also take care when selecting questions that could be

offensive in the local cultural context. Questions of a personally sensitive nature, or which may be

deemed culturally offensive, are likely to antagonize participants. This could have a negative effect

upon both response rates, and the nature of feedback to open-response questions (Cohen et al., 2011,

pp. 395-396). The wording of questions related to peer-interaction must also be chosen carefully, as

some respondents may view admission of peer-interaction during in-class activities as an admission of

academic dishonesty. Questions designed to gauge impacts on student-student interaction must avoid

giving the impression that such interaction is inherently undesirable.

The use of online surveys is also an attractive option for collecting feedback from classroom

teachers. While many of the same considerations outlined above also apply to the design of a teacher

feedback questionnaire, issues related to literacy and cultural sensitivities are to some degree mitigated.

However, care must still be taken to avoid the use of negative wording, or wording that might lead

respondents to select a more desirable, as opposed to accurate response (Cohen et al., 2011, pp. 396-

397). The inclusion of a higher number of open-response questions may be desirable, as they allow

more leeway for the teachers to share expert insights into the successes (or drawbacks) of the RLO

designs (p. 382). The use of online surveys with both students and teachers can also enable the

researcher to identify potential themes or issues that could form the basis of more focused interviews

with selected participants.

Interviews are a third data collection strategy that would be appropriate in pragmatic DBR, as

well as in the current context. Among the myriad interview scenarios available are the one-on-one

interview between the researcher and a participant, and the focus group approach (Cohen et al., 2011,

pp. 412-415). Interviews can follow a prescribed schedule, or they can be semi-structured (using a set of


prescribed questions, but allowing for the pursuit of unexpected tangents as comments, themes or

issues arise), or purely unstructured (allowing the participant to discuss issues without being guided by a

researcher’s agenda) (pp. 416-417). Interviews provide a greater voice to individual participants, which

could be especially useful in the current context where the writing skills of the EFL students may limit

their responses to open-response survey questions. They allow for deeper investigation of participant

experiences and attitudes than afforded through written survey responses. However, there are serious

considerations. One of these is the issue of power imbalance between the researcher and the

participant student (pp. 421-426, 442-443). As previously noted, local culture prescribes the avoidance

of conflict and the saving of face. In a one-on-one interview, students may be reluctant to express their

true viewpoints if they perceive that information as undesirable or embarrassing to either themselves or

the researcher. Special care must be taken to ensure that participants are comfortable with expressing

their true viewpoints. This can be achieved through careful attention to the researcher’s professional

demeanor, body language, and appearance of genuine interest (pp. 422-424). This can also be achieved

through careful selection of the questions for the interview schedule (Cohen et al., 2011, pp. 416-421;

Roulston, 2011). Easy to answer, factual response questions are recommended to build rapport, and to

lead into more open-response, attitude-probing questions. If a question is likely to be perceived as

either too personal, or potentially generating conflict or embarrassment, then the researcher should

attempt to ask more roundabout questions to elicit the same information.

Implementing interviews requires consideration of a number of practical issues. One issue

relates to participant selection. In its current iteration, the QR Cache project is focusing on a relatively

small sample size of students enrolled in three course sections. Consideration must be given not only to

how many interviews would be appropriate, but also to securing the participation of a sufficient number

of students to generate truly representative and generalizable themes from the responses (Cohen et al.,

2011, pp. 143-149, 421; Onweugbuzie & Collins, 2007). This problem is further compounded by time

constraints. The current iteration of the research takes place during an eight-week intersession term,

and the students are required to take a full daily schedule of classes. There may be insufficient time for

either students or teachers to commit to participating in interviews. Another practical issue relates to

determining what and how much to observe and record, and how to capture that data (Cohen et al.,

2011, pp. 426-427). Interviews allow respondents to answer questions and provide opinions in their

own words. But participants also express themselves through body language, and those non-verbal cues

can provide a wealth of insight (pp. 426-427). However, such data is often lost by inexperienced

researchers who focus too much attention on verbal responses. The use of audio recording instruments

could ensure that no spoken responses are missed, and allow the researcher to pay more attention to

both participant body language and their own appearance of genuine interest. Video recording

equipment could allow the researcher to capture both verbal and non-verbal data. However, the use of

recording media may have a distorting effect upon the comfort level of the participants, and their

responses (pp. 421-426). In the local cultural context, some potential participants (particularly females)

may outright refuse to participate if video recording equipment is to be used (Cohen et al., 2011, p. 424;

Qatar Visitor, 2000). Even in situations where recording equipment is used, data may be lost or

distorted in the process of transcription.


With respect to interviewing with participant teachers, this option could elicit a great deal of

professional insight into the effectiveness of the RLO designs, and the types of interactions generated.

From a pragmatic perspective, these insights could be invaluable guides for improvements in RLO design

and their use in similar classroom contexts. However, there are some potential drawbacks. One of

these is the nature of the relationship between the teacher-researcher and the other participant

teachers (Cohen et al., 2011, pp. 421-426; Dwyer & Buckle, 2009; Huckaby, 2011). Colleagues may

either feel uncomfortable participating in a formal interview with “one of their own,” or they may feel

too comfortable. In the latter scenario, it may be difficult to “draw the line” between useable responses

and more casual (perhaps even “off-the-record”) information. Dwyer and Buckle (2009) also caution

that participant colleagues may even limit the amount of detail in their responses because they “may

make assumptions of similarity and therefore fail to explain their individual experiences fully” (p. 58). It

may prove difficult or uncomfortable for a peer, acting as an interviewer, to probe more deeply without

creating either tension or embarrassment.

Given the time constraints required to design, conduct, transcribe and analyze effective

interviews, this data collection option is not the most desirable strategy during the current iteration of

the QR Cache project. However, data collected through the use of tests and surveys during the current,

pilot phase could be used to guide the design of appropriate interview schedules for future iterations

drawing upon expanded time frames and student sample sizes.

Proposed Data Collection Strategy

For the current phase of the QR Cache project, data will be collected through the use of tests

and online questionnaires. The tests to be used consist of the Practical Evaluations (PEs) and

Knowledge-Based Tests (KBTs) already employed to cover the Hardware Components unit of the target

course. For this phase, only data on successful completion rates, and number of attempts per student,

will be gathered. Participating teachers will submit this information in an Excel spreadsheet without

identifying individual students. Data will only be included for students who have signed an informed

consent form.

An online questionnaire will be used to gather data on student interactions with the RLOs, their

peers, and teachers. Questions relating to demographic information will not be incorporated due to the

relative homogeneity of the participant groups. The questionnaire will be password-protected, and the

password will be entered by a research assistant. This will ensure that the questionnaire is only

completed by consenting participants, and that each student only completes the questionnaire once

(Cohen et al., 2011, pp. 77-81; Kanuka & Anderson, 2007). A variety of fixed and open-response

question types will be used, covering such themes as technical accessing of the RLOs, content

presentation, peer and teacher interaction, and overall impressions. A similarly structured and themed

online questionnaire will be developed for teacher feedback. The questionnaires will be hosted on

QUIA, which is a subscription-based online service providing secure learning management system-like

tools including customizable feedback surveys (QUIA, 2012).


If the QR Cache project does proceed to a second DBR iteration, then interviews with participant

teachers could be developed. A semi-structured focus group interview would be used, with a schedule

of questions derived from the themes arising from the analysis of the testing and questionnaire data

(Cohen et al., 2011, pp. 432-433). This would be used for member verification of the themes to be

addressed in the RLO redevelopment, and the identification of potential course topics for which new

RLOs could be deployed in subsequent research iterations.

The data collection strategy for the current phase of the QR Cache project is illustrated in Figure

1, below:

Figure 1: Proposed Data Collection Strategy for the QR Cache Research Project

Data Analysis Procedures

Data on student performance in the PE and KBT assessments, submitted by classroom

instructors in Excel spreadsheet format, will be collated into a single master spreadsheet. These data

will then be saved in a comma delimited (.csv) format, and imported into SPSS. SPSS will be used to

perform statistical calculations, including summaries of the overall pass and failure rates, and average

number of assessment attempts, for each course section. Parametric tests of significant difference are


considered especially strong statistical support in hypothesis testing, and can be drawn upon in this

context (Changing Minds, 2012; Cohen et al., 2011, p. 606; Lunsford & Lunsford, 1996; Motulski, 1995).

While the sample size is relatively small, and not truly representative of the general populations of

either the region (Qatar) or all students at CNA-Q, the test data can be considered parametric

(representative of a broad population). This is because of the relative homogeneity of students in the

Technical Preparatory Program (TPP), and the standardization of the assessment instruments used for

all students in that program. Analysis of variance (ANOVA) calculations will be performed to determine

significant differences between pass, failure, and average attempt number rates across and between the

control and experimental groups. ANOVA tests of significance differences are “premised on…random

sampling, normal distribution of scores and parametric data, and…can be used with three or more

groups” (Cohen et al., 2011, p. 645). The results will be used to determine if either instructional method

(RLOs or course workbooks) leads to higher rates of successful assessment completion, or fewer average

numbers of attempts. However, due to the sample size, it will be difficult to place too much confidence

in the generalizability of the findings (Cohen et al., 2011, p. 144; Onweugbuzie & Collins, 2007;

Stockburger, 1996). The utility of these calculations will be as a tentative illustration of the effectiveness

of the RLO designs compared to other teaching methods, and as a point of comparison if similar data is

collected in subsequent iterations of the research.

Two types of analysis will be required for data collected via the online questionnaires. The

actual responses will be exported from QUIA into a comma delimited format, which will then be

imported into an Excel spreadsheet. Fixed and open-response questions will be isolated and separated

into different spreadsheet files. The fixed-response questions will be coded, as appropriate, based upon

response type (Cohen et al., 2011, p. 428). The coded data will be resaved and imported into SPSS. SPSS

will be used to generate frequency distributions of responses, and to compare the frequency

distributions across thematically-related questions (pp. 611-615). These will be examined for the

significance of the levels of responses that would indicate different types of interaction (activity). These

frequencies will be plotted against the corresponding domains of the FRAME model (Koole, 2009), and

examined within the framework of Transactional Distance Theory (TDT) (Moore, 1989, 1991). This

analysis will provide an understanding of the effectiveness of the RLOs at reducing specific types of

transactional distance, and at optimizing the types of activities delineated by the FRAME model as

indicators of effective m-learning design.

The responses to open-response questions will be grouped by specific question. These will be

initially read, in their entirety, to develop overall impressions of the themes represented in the

responses (Cohen et al, 2011, pp. 565-567). A tentative list of themes will be developed, along with a

corresponding coding system, to be used as a guide for a second reading. At this stage, the coding

system will be used to code the responses by theme, and memos will be used to record the researcher’s

impressions and insights (pp. 559-563). These memos and transcription codings will be used to refine

(and, if necessary, expand) the coding system (p. 557). Transcript coding will then be repeated to more

accurately identify illustrations of the themes. Frequency counts will be performed for the coded data

to determine the frequencies of responses reflecting each theme (p. 554). The resultant calculations will

be compared to the analysis of the data from the fixed-response questions. This is intended to lend


further support to the initial conclusions about effects on transactional distance, to provide further

insight into how these effects came about, and to determine the degree to which the RLOs reflect the

key elements of the FRAME model. Individual quotations could be excerpted for the research report, to

add weight to the data analysis and conclusions. The coded transcript analysis would also be used to

identify problems with the RLO designs that would need to be addressed in future DBR iterations in

order to further reduce transactional distance, and improve the overall m-learning design.

Additional Practical, Ethical and Philosophical Issues

With any proposed research design, there are practical, ethical and philosophical issues that will

need to be considered. A number of these issues have already been highlighted in the exploration of

alternative data collection methods. Examples of practical issues include the development, piloting, and

deployment of tests and online questionnaires, as well as local time constraints. In terms of ethical

issues, informed consent and anonymity have already been mentioned. Philosophically, the concepts of

power and relationship issues, researcher roles, bias and reflexivity, and reliability and validity have

been discussed. Some of these issues warrant further elaboration, and there are additional issues that

must be considered.

The practical issues of instrument development and time constraints have already been

discussed in detail. However, additional concerns must be addressed. One major concern relates to

data security for the online questionnaires (which also represents an ethical concern) (Cohen et al.,

2011, p. 94; Kanuka & Anderson, 2007). The questionnaires will be hosted on a stable, password-

protected service. The results will be exported and stored (including backups) on a password-protected

local server. This will minimize the risk of data loss, and ensure that only authorized personnel have

access to the data.

Questionnaire response rates are a second practical concern. Where open invitations to

complete postal surveys are sent to participants, allowing them to complete such questionnaires at their

convenience, “response rates can be as low as 20-30 per cent, and this compromises the reliability of

the data very considerably” (Cohen et al., 2011, p. 263). Higher initial response rates can be expected

from online questionnaires, which “tend to have lower item non-response than paper-based surveys”

(p. 261). However the structure and wording may have an impact on completion rates for the entire

survey. Fowler (2009) suggests that subsequent follow-ups can lead to between a further “quarter and

a third of people [agreeing] to complete a survey” (as cited in Cohen et al., 2011, p. 263). However,

given the small sample size, it is desirable to maximize the response rate as much as possible. To

facilitate this, class time will be allocated for questionnaire completion (p. 262). A research assistant will

invigilate the session, and use a roster of consenting participants to monitor who completes the survey.

This will also ensure that students who are absent can be provided with a second opportunity in a

subsequent class.

A third practical concern relates to the authenticity of the research setting. Anderson and

Shattuck (2012) note that one of the hallmarks of DBR is that it is


situated in a real educational context provid[ing] a sense of validity to the research and

ensur[ing] that the results can be effectively used to assess, inform, and improve practice in at

least this one (and likely other) contexts (p. 16).

For this iteration, time and resource constraints do not permit access to an authentic petrochemical

refinery distributed control center (DCS) (the type of setting in which TPP graduates will be working). A

second iteration of the QR Cache project may be provided with access to a fully functional DCS mockup

on campus at CNA-Q. That facility could be used for a “walk-and-scan” tour (where students would scan

QR codes on various equipment samples to learn their names and functions in English. For the current

iteration, samples of typical workplace computer components will be placed throughout the regular

course laboratory for students to explore using their mobile devices. While this does not authentically

simulate the students’ eventual workplace, it is situated in a “real educational context” (p. 16), and

provides access to authentic equipment (which would normally only be depicted in graphics in the

course workbook).

A major ethical issue in any research project is informed consent (Cohen et al., 2011, pp. 77-81;

Kanuka & Anderson, 2007). All potential participants will be provided with a detailed briefing on the

nature of the research, and the activities in which they will be participating. They will also be asked to

sign an informed consent letter, drafted and approved under CNA-Q Department of Applied Research

protocols. Copies of the signed consent forms will be returned to students for their records. A reminder

of the informed consent, and the option to withdraw from participation, will be included on the

“Welcome” page for the online questionnaire (Kanuka & Anderson, 2007). Additionally, any potential

for undue coercion of potential participants will be mitigated by ensuring that the researcher is not also

assigned as the classroom teacher for either of the course sections that will be using the RLOs (Cohen et

al., 2011, pp. 80, 88-99).

Related to the issue of informed consent is the concept of beneficence, or non-maleficence.

This Social Sciences and Humanities Research Council of Canada (1981) described this concept as

holding that

greater consideration must be given to the risks to physical, psychological, humane, proprietary

and cultural values than to the potential contribution of research to knowledge (as cited in

Cohen et al., 2011, p. 85).

In the QR Cache project, it is not anticipated that participation will harm students in any way. In the

event that students fail to pass the PE or KBT assessments, participating teachers have agreed to provide

remedial instruction using the approved course workbook. Arrangements will also be made to re-

administer the PEs and KBTs without academic penalty. It is also expected that participants will benefit

from the research in terms of a more enjoyable learning experience, and the development of soft skills

that will help them in their eventual workplace (Traxler & Wishart, 2011).

Confidentiality is an ethical issue that is closely related to informed consent (Cohen et al., 2011,

pp. 77-81; Kanuka & Anderson, 2007). This has already been discussed in relation to the collection of PE

and KBT performance data. The deployment of the online questionnaire also provides for participant


confidentiality. First, neither the classroom teacher nor the researcher will be present when students

complete the survey. Second, the questionnaire is not dependent upon user login credentials, nor are

demographic data solicited. Thus, no metadata are collected which could be used to trace responses to

individual participants.

Sample selection has practical, ethical and philosophical implications. Practically, the current

iteration uses a convenience sampling strategy (Cohen et al., 2011, pp. 155-156; Onweugbuzie & Collins,

2007). Students are to be selected from existing course sections. Ethically, this practice excludes

potential participants from the wider college population (Cohen et al., 2011, pp. 161-162; Onweugbuzie

& Collins, 2007). However, all students in the current TPP cohort do have an equal opportunity to

participate (or to decline). Philosophically, the sample selection strategy will have an impact upon

validity and reliability (Cohen et al., 2011, pp. 180-183; Onweugbuzie & Collins, 2007). The current

iteration focuses on a specific sub-population (TPP students), who will be studying the same content and

assessed with the same instruments. As previously discussed, these PE and KBT instruments have

already been vetted for validity and reliability. While this focus will make it difficult to generalize results

across the broader college population, they could be generalized across students in the TPP program.

The sample size also raises questions of validity and reliability, as it is difficult to generalize conclusions

from statistical calculations performed on small samples. However, from the perspective of DBR, the

results may be sufficient to develop tentative hypotheses, and to identify themes to address in RLO

redevelopment and future deployment. They could serve as a starting point for more in-depth study

with a broader participant base, leading to eventual theoretical and practical conclusions and policy

developments (Anderson & Shattuck, 2012; DBRC, 2003).

Validity and reliability also need to be considered in the coding of questionnaire transcripts.

Researcher bias may impact upon the interpretation of responses to open questions (Cohen et al., 2011,

p. 540). Using multiple raters to code the transcripts could reduce this effect (p. 556). Coded transcripts

could be cross referenced, and an interrater reliability coefficient calculated (Stemler, 2004). Such

measures would increase the level of confidence in the identification of response themes.

The use of online questionnaires, as opposed to interviews, could be regarded as a lost

opportunity to develop a deeper understanding of students’ interactions with the RLOs (Akbayrak, 2000;

Cohen et al., 2011, pp. 377, 409). The practical basis of this decision has already been noted, as has the

potential to use themes emerging from the questionnaire analysis to develop interview schedules for

future DBR iterations.

Researcher roles and relationships with participants present ethical and philosophical issues

(Cohen et al., 2011, pp. 232-234). The effects of perceived power imbalances upon the behaviors and

responses of participants have already been discussed, as have measures to minimize these effects for

the online questionnaires. Cultural influences on the provision of candid responses have also been

alluded to. However, such potential influence points to a need to consider insider/outsider research

issues (Cohen et al., 2011, pp. 232-234; Dwyer & Buckle, 2009; Huckaby, 2011). Differences in ethnic

and cultural backgrounds between the researcher and participants may amplify perceived power

imbalances and willingness to provide forthright responses.


Dwyer and Buckle (2009) and Huckaby (2011) offer insights into how to handle the intricacies of

insider/outsider relations. Dwyer and Buckle point out that even when a researcher is not a member of

the researched group, personal and professional background and subject matter expertise likely places

him/her somewhere in between being either a true insider or outsider. Thus, the researcher should be

respectful of not being a true insider, but should also be reflexive to personal influences on relations

with participants and interpretations of responses. Huckaby, having experienced a reversal of typical

power imbalances while interviewing more established academics, offers suggestions valid in any

research context. She advises researchers to be as informed as possible about participants’

backgrounds going into the research, but not to assume anything about them. She extolls the

importance of taking as much time as possible to establish rapport and mutual respect. She also advises

letting the participants’ responses guide the research process, and being reflexive to their viewpoints.

But she also cautions researchers not to lose sight of their research intentions, noting that she waited

for appropriate opportunities to return her investigative efforts to their primary path. While both

Dwyer and Buckle (2009) and Huckaby (2011) offer advice from the perspective of interview research,

their suggestions remain valid when focusing on surveys in the current context.

Developing the QR Cache project as DBR presents another major philosophical issue. That is,

the overall utility of both the intervention and the research results. As previously discussed, DBR aims

to produce practical results that could also be applied in similar contexts, as well as contribute to overall

understandings of educational theory. The research methodologies, data collection and analysis

strategies, and the practical, ethical and philosophical issues that have been explored have all focused

on these overarching pragmatic purposes. The QR Cache project aims to fulfill local stakeholder needs,

as well as uncover potential improvements to the RLO designs. Through future iterations, the RLOs

could be more widely integrated into local practices, and contribute to a greater understanding of m-

learning theory and effective m-learning design.

Conclusions

The context and aims of the QR Cache research project fit within the paradigm of pragmatic

research. DBR is an ideal methodological approach, which has been used to guide the selection of

existing tests and online questionnaires as data collection tools. Participant interviews have also been

explored as a potential tool for future DBR iterations. The specific data collection and analysis strategy

has been outlined along with the practical, ethical and philosophical issues that must be addressed

during the planning, implementation, and data analysis phases. All of these elements have been

examined with the pragmatic objectives of producing an intervention that will be of immediate utility,

and form the foundation of iterative improvements. They also aim to inform the development of an

effective local m-learning strategy, and contribute to the body of knowledge about m-learning theory

and design.


References

Akbayrak, B. (2000). A comparison of two data collection methods: Interviews and questionnaires.

Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 18, 1-10. Retrieved from

http://www.efdergi.hacettepe.edu.tr/200018BURCU%20AKBAYRAK.pdf

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research?

Educational Researcher, 41(1), 16-25. DOI: 10.3102/0013189X11428813. Retrieved from

http://edr.sagepub.com/content/41/1/16.full

Changing Minds (2012). Parametric vs. non-parametric tests. Retrieved from

http://changingminds.org/explanations/research/analysis/parametric_non-parametric.htm

Clearview (n.d.). Cultural awareness: Middle East cultural overview. Retrieved from

http://www.clearviewqatar.com/culturalawareness.html

Cohen, L., Manion, L & Morrison, K. (2011). Research methods in education (7th ed). New York:

Routledge.

College of the North Atlantic-Qatar (2011). Strategic plan 2011-2016: Creating the future…together.

Doha: College of the North Atlantic-Qatar.

Denzin, N. K. (2008). The new paradigm dialogs and qualitative inquiry. International Journal of

Qualitative Studies in Education, 21(4), 315-325. Retrieved from

http://dx.doi.org/10.1080/09518390802136995

The Design-Based Research Collective (2003). Design-based research: An emerging paradigm for

educational inquiry. Educational Researcher, 32(1), 5-8. Retrieved from

http://www.designbasedresearch.org/reppubs/DBRC2003.pdf

Dwyer, S. & Buckle, J. (2009). The space between: On being an insider-outsider in qualitative research.

International Journal of Qualitative Methods, 8(1), 54-63. Retrieved from

http://ejournals.library.ualberta.ca/index.php/IJQM/article/view/2981/5198

Flyvbjerg, B. (2004). Five misunderstandings about case study research. In Seale, M., Gobo, G., Gubrium,

J. & Silverman, D. (Eds.), Qualitative research practice, 420-434. Thousand Oaks, CA: Sage.

Retrieved from http://poli.haifa.ac.il/~levi/res/fivemisunder.pdf

Gerring, J. (2004). What is a case study and what is it good for? American Political Science Review (98)2,

341-354. Retrieved from

http://people.ucalgary.ca/~nmstuewe/CaseStudy/pdf/whatisacasestudy.pdf

Huckaby, M. (2011). Researcher/researched: Relations of vulnerability/relations of power. International

Journal of Qualitative Studies in Education, 24(2), 165-183. DOI:

10.1080/09518398.2010.529851. Retrieved from http://0-

www.tandfonline.com.aupac.lib.athabascau.ca/doi/pdf/10.1080/09518398.2010.529851


Kanuka, A., & Anderson, T. (2007). Ethical issues in qualitative e-learning research. International Journal

of Qualitative Methods, 6(2), 20-39. Retrieved from


Kaptelinin, V., & Nardi, B. (2006). Acting with technology: Activity theory and interaction design.

Cambridge, MA: MIT Press.

Koole, M. L. (2009). A model for framing mobile learning. In M. Ally (Ed.), Mobile learning: Transforming

the delivery of education and training, 25-47. Edmonton, AB: AU Press. Retrieved from

http://www.aupress.ca/index.php/books/120155

Kozma, R. (1994a). Will media influence learning? Reframing the debate. Educational Technology

Research and Development, 42(2), 7-19.

Kozma, R. (1994b). A reply: Media and methods. (1994). Educational Technology Research and

Development, 42(3), 11-14.

MacDonald, C., Stodel, E., Muirhead, B. & Thompson, T. L. (n.d.). Conceptual Frameworks Learning

Object. Retrieved from http://innovation.dc-

uoit.ca/conceptualframeworks/CF_LO_content.html

Lunsford, T. & Lunsford, B. (1996). Parametric data analysis. Journal of Prosthetics and Orthotics, 8(2),

65-76. Retrieved from http://www.oandp.org/jpo/library/1996_02_065.asp

MacLeod, C. (2011). CNA-Q students and faculty: Report on learning technologies usage. Unpublished

report, College of the North Atlantic-Qatar, Doha, Qatar. Retrieved from http://cna-

qatar.libguides.com/content.php?pid=196496&sid=1648461

Metodieva, V. (2012, February 3). Qatar’s smartphone market Q4 2011: Executive Summary. Retrieved

from

http://www.ictqatar.qa/sites/default/files/documents/Qatar%20Smartphone%20Market%20-

%20Q4%202011.pdf

Moore, M. (1989). Three types of interaction. The American Journal of Distance Education, 3(2), 1-6.

Moore, M. (1991). Editorial: Distance education theory. The American Journal of Distance Education,

5(3), 1-6. Retrieved from http://www.ajde.com/Contents/vol5_3.htm#editotial

Motulski, H. (1995). Intuitive biostatistics: Choosing a statistical test. Retrieved from

http://www.graphpad.com/www/book/choose.htm

Nagy, M. (2012, March 27). Where have mobile apps gone. Digital Qatar. [Web log comment]. Retrieved

from http://www.digitalqatar.net/2012/03/27/where-have-mobile-apps-gone/

Newbury, J. (2011). A plan for theoretical inconsistency. International Journal of Qualitative Methods,

10(4), 335-347. Retrieved from http://0-


web.ebscohost.com.aupac.lib.athabascau.ca/ehost/pdfviewer/pdfviewer?sid=8ea169ed-9955-

4755-bd10-b67ef37345c5%40sessionmgr14&vid=2&hid=14

Onwuegbuzie, A. & Collins, K. (2007). A typology of mixed methods sampling designs in social science

research. The Qualitative Report, 12(2), 281-316. Retrieved from

http://www.nova.edu/ssss/QR/QR12-2/onwuegbuzie2.pdf

Power, R. (2012a). Developing a conceptual framework for the QR cache research project. Unpublished

manuscript, Centre for Distance Education, Athabasca University, Athabasca, Canada.

Power, R. (2012b). Examining theoretical paradigm issues in the QR cache m-learning research project.

Unpublished manuscript, Centre for Distance Education, Athabasca University, Athabasca,

Canada.

Power, R. (2012c). QR Cache: Assessing the feasibility of using Quick Response (QR) codes and mobile

devices to help students master computer-related workplace English competencies. Research

proposal submitted to the Research Ethics Committee, College of the North Atlantic-Qatar,

Doha, Qatar.

Qatar Academy (n.d.). Cultural competency guidelines. Qatar Academy staff handbook. Retrieved from

http://qapstaffhandbook.qataracademy.wikispaces.net/Cultural+guidelines

Qatar Visitor (2000). Essential do’s and don’ts for Qatar living. [Web log comment]. Retrieved from

http://www.qatarvisitor.com/index.php?cID=412&pID=1386

QUIA (2012). About QUIA. Retrieved from http://www.quia.com/findout.html

Rost, M. (2006). Generating student motivation. Retrieved from

http://www.pearsonlongman.com/ae/worldview/motivation.pdf

Roulston, K. (2011). Working through the challenges in doing interview research. International Journal of

Qualitative Methods, 10(4), 348-366. Retrieved from


Stemler, S. E. (2004). A comparison of consensus, consistency, and measurement approaches to

estimating interrater reliability. Practical Assessment, Research & Evaluation, 9(4). Retrieved

from http://PAREonline.net/getvn.asp?v=9&n=4

Stockburger, D. W. (1996). Introductory statistics: Concepts, models and applications (WWW version 1.0,

Rev. ed.). Retrieved from http://www.psychstat.missouristate.edu/introbook/sbk00.htm

Traxler, J. & Wishart, J. (2011). Making mobile learning work: Case studies of practice. Bristol: UK:

ESCalate. Retrieved from http://escalate.ac.uk/8250

Trochim, M. K. (2006). Likert scales. In Research knowledge database. Retrieved from:

http://www.socialresearchmethods.net/kb/scallik.php


Warraich, K. & Dahlstrom, E. (2012, February). A profile of student mobile computing in Education City.

Presented at the Technology in Higher Education Conference 2012, Doha, Qatar. Retrieved from

http://technology.qatar.tamu.edu/2885.aspx

Planning Research Strategies for the QR Cache Research...

Documents

Transcript of Planning Research Strategies for the QR Cache Research...