Q1. Discuss eight variables which are commonly used in research (20 Marks)A VARIABLE is a measurable characteristic that varies. It may change from group to group, person to person, or even within one person over time. The common variable types are:i. Dependent variablesShow the effect of manipulating or introducing the independent variables. For example, if the independent variable is the use or non-use of a new language teaching procedure, then the dependent variable might be students' scores on a test of the content taught using that procedure. In other words, the variation in the dependent variable depends on the variation in the independent variable.ii. Independent variables. . . are those that the researcher has control over. This "control" may involve manipulating existing variables (e.g., modifying existing methods of instruction) or introducing new variables (e.g., adopting a totally new method for some sections of a class) in the research setting. Whatever the case may be, the researcher expects that the independent variable(s) will have some effect on (or relationship with) the dependent variables.iii. Intervening variables. . . refer to abstract processes that are not directly observable but that link the independent and dependent variables. In language learning and teaching, they are usually inside the subjects' heads, including various language learning processes which the researcher cannot observe. For example, if the use of a particular teaching technique is the independent variable and mastery of the objectives is the dependent variable, then the language learning processes used by the subjects are the intervening variables.iv. Moderator variables. . . affect the relationship between the independent and dependent variables by modifying the effect of the intervening variable(s). Unlike extraneous variables, moderator variables are measured and taken into consideration. Typical moderator variables in TESL and language acquisition research (when they are not the major focus of the study) include the sex, age, culture, or language proficiency of the subjects.v. Control variablesLanguage learning and teaching are very complex processes. It is not possible to consider every variable in a single study. Therefore, the variables that are not measured in a particular study must be held constant, neutralized/balanced, or eliminated, so they will not have a biasing effect on the other variables. Variables that have been controlled in this way are called control variables.

vi. Extraneous variables. . . are those factors in the research environment which may have an effect on the dependent variable(s) but which are not controlled. Extraneous variables are dangerous. They may damage a study's validity, making it impossible to know whether the effects were caused by the independent and moderator variables or some extraneous factor. If they cannot be controlled, extraneous variables must at least be taken into consideration when interpreting results.

Q2. While using relevant examples, explain four types of reliability and four types of validity commonly used in research(40MKs)a) Reliability has to do with the quality of measurement. In its everyday sense, reliability is the "consistency" or "repeatability" of measures used in research methods.Joppe(2000) defines reliability as the extent to which results are consistent over time and an accurate representation of the total population under study is referred to as reliability and if the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable. (p 1)ValidityJoppe(2000) provides the following explanation of what validity is in quantitative research:Validity determines whether the research truly measures that which it was intended to measure or how truthfulthe research results are. In other words, does the research instrument allow you to hit the bulls eye of your research object? Researchers generally determine validity by asking a series of questions, and will often look for the answers in the research of others. (p. 1)'Reliability' of any research is the degree to which it gives an accurate score across a range of measurement. It can thus be viewed as being 'repeatability' or 'consistency'. In summary:Inter-rater: Different people, same test.Test-retest: Same people, different times.Parallel-forms: Different people, same time, different test.Internal consistency: Different questions, same construct.i. Inter-rater or inter-observer reliability(Different people same test)This is a kind of reliability that is at issue when the same stimulus (person, event, behavior, etc.) must be rated by more than one rater. Eg. In studies of the relationship between physical attractiveness and social development, the researchers need to know how attractive the person is. (Research of this kind asks questions such as, do prettier people develop better social skills?) How can this rating be done? Calculate the ratio of length of nose to distance between the ears? While some such physical indexes of attractiveness have been developed, the most common way is to assemble a panel of judges to rate the stimuli. The researcher looks at the extent to which the raters agree on their ratings. When inter-rater reliability is low, the researcher has to wonder if it is possible to classify persons on a dimension such as attractiveness or whether his or her attempts to do so have failed. Inter-rater reliability is measured in several ways, such as the percentage of agreement among the judges or a correlation.ii. Test-retest reliability (Same people, different times).Test-retest reliability refers to whether or not the sample you used the first time you ran your survey provides you with the same answers the next time you run an identical survey. There are factors that may affect their answers, but presumably if a survey is taken twice and nothing has changed to affect responses, the results should be the same.The amount of time allowed between measures is critical. The shorter the time gap, the higher the correlation; the longer the time gap, the lower the correlation. This is because the two observations are related over time the closer in time we get the more similar the factors that contribute to error.Instruments such as IQ tests and surveys are prime candidates for test-retest methodology, because there is little chance of people experiencing a sudden jump in IQ or suddenly changing their opinions.On the other hand, educational tests are often not suitable, because students will learn much more information over the intervening period and show better results in the second test.iii. Parallel-forms: Different people, same time, different test, different persons simultaneously.This refers to the ability of similar questions asking about the same item to have similar/near identical responses. You want to make sure that all questions relating to a particular item are being read as equal by the same sample, and thus generating similar results. A researcher does this by creating a list of questions that are all asking about the same things, then randomly dividing them up into separate tests with the same sample, and seeing the correlation between the results.iv. Internal consistency: Different questions, same construct.It refers to the respondents ability to answer similar questions within a single survey the same. Such as the answer to: How much do you like this product and comparing it to On a scale of 1 to 10, what are your general feelings about this product. There are a lot of ways to judge internal consistency, but the goal is to make sure that questions asking similar questions yield similar responses.

b) Validity refers to how well a test measures what it is purported to measure. Types of Validityi. Face Validity ascertains that the measure appears to be assessing the intended construct under study. The stakeholders can easily assess face validity. Although this is not a very scientific type of validity, it may be an essential component in enlisting motivation of stakeholders. If the stakeholders do not believe the measure is an accurate assessment of the ability, they may become disengaged with the task.Example: If a measure of art appreciation is created all of the items should be related to the different components and types of art. If the questions are regarding historical time periods, with no reference to any artistic movement, stakeholders may not be motivated to give their best effort or invest in this measure because they do not believe it is a true assessment of art appreciation. ii. Construct Validity is used to ensure that the measure is actually measure what it is intended to measure (i.e. the construct), and not other variables. Using a panel of experts familiar with the construct is a way in which this type of validity can be assessed. The experts can examine the items and decide what that specific item is intended to measure. Students can be involved in this process to obtain their feedback.Example: A womens studies program may design a cumulative assessment of learning throughout the major. The questions are written with complicated wording and phrasing. This can cause the test inadvertently becoming a test of reading comprehension, rather than a test of womens studies. It is important that the measure is actually assessing the intended construct, rather than an extraneous factor.iii. Criterion-Related Validity is used to predict future or current performance - it correlates test results with another criterion of interest. Example: If a physics program designed a measure to assess cumulative student learning throughout the major. The new measure could be correlated with a standardized measure of ability in this discipline, such as an ETS field test or the GRE subject test. The higher the correlation between the established measure and new measure, the more faith stakeholders can have in the new assessment tool.iv. Formative Validity when applied to outcomes assessment it is used to assess how well a measure is able to provide information to help improve the program under study.Example: When designing a rubric for history one could assess students knowledge across the discipline. If the measure can provide information that students are lacking knowledge in a certain area, for instance the Civil Rights Movement, then that assessment tool is providing meaningful information that can be used to improve the course or program requirements.v. Sampling Validity (similar to content validity) ensures that the measure covers the broad range of areas within the concept under study. Not everything can be covered, so items need to be sampled from all of the domains. This may need to be completed using a panel of experts to ensure that the content area is adequately sampled. Additionally, a panel can help limit expert bias (i.e. a test reflecting what an individual personally feels are the most important or relevant areas).Example: When designing an assessment of learning in the theatre department, it would not be sufficient to only cover issues related to acting. Other areas of theatre such as lighting, sound, functions of stage managers should all be included. The assessment should reflect the content area in its entirety.

Q3. Differentiate between reliability and validity as used in research (10Mks)Reliability A study or test's reliability is the measure of whether the same results can be obtained given differing circumstances. For example, having a reliable test means a student will score the same or close to the same score on the test under different circumstances. If five different people graded the test, the scores should be the same or similar. If the student took the test on a Monday morning or a Wednesday evening, it shouldn't affect the scores. In addition, someone scoring the test on Monday and then again on Friday should not arrive at different scores. Reliability controls the results of a test or study so external factors -- such as when or by whom it is conducted -- do not change the results.Validity A study or test's validity is the measure of whether the results actually relate to what the test or study developer intended to measure or test. If a test claims to evaluate a student's ability to use multiplication, for example, but consists only of subtraction questions, it is not valid. If a study was investigating the mating calls of a certain bird but only investigated male birds, for example, or recorded calls outside of mating season, it would also be invalid. Examining a study's validity allows you to decide whether the results should be accepted and used to generalize or not.

Page 3

Ketter Robert